Scheduling on Heterogeneous Hadoop System in Eucalyptus Private Cloud

Cloud Computing is emerging as a new computational paradigm shift. Hadoop-Map Reduce has become a powerful computation model for processing large data on distributed commodity hardware clusters such as clouds. A large number of (heterogeneous) clients use the same Hadoop cluster. Map Reduce (MR) has become a popular programming model for running data intensive application on the cloud. Completion time goals or deadline of map reduce jobs set by users are becoming crucial in existing cloud based data processing environments such as Hadoop. There is a conflict between the scheduling MR jobs to meet deadline and data locality by assigning tasks to nodes that contain their input data in public cloud environments like Amazon web services. To meet a deadline, a task maybe scheduled on a node without local input data for that task causing expensive data transfer from a remote node. One of the major requirement in cloud computing is related to optimizing the resource being allocated with the objective of minimizing the costs associated with it. The research proposes an approach to schedule jobs on scalable and heterogeneous hadoop system in cloud environments with respect to the number of incoming jobs and available resources. A scheduler is proposed to address the problems which are primarily based on an efficient resource allocation and workload management strategies for large scale cloud environments. It deals with effective resource allocation strategies for achieving user satisfaction and maximizing the profit for cloud service providers.