Invention Grant
US09411645B1 Scheduling MapReduce tasks based on estimated workload distribution
有权
基于估计的工作负载分配计划MapReduce任务
- Patent Title: Scheduling MapReduce tasks based on estimated workload distribution
- Patent Title (中): 基于估计的工作负载分配计划MapReduce任务
-
Application No.: US14982296Application Date: 2015-12-29
-
Publication No.: US09411645B1Publication Date: 2016-08-09
- Inventor: Ning Duan , Liya Fan , He Yuan Huang , Jian Wang , Zhi Hu Wang
- Applicant: International Business Machines Corporation
- Applicant Address: US NY Armonk
- Assignee: International Business Machines Corporation
- Current Assignee: International Business Machines Corporation
- Current Assignee Address: US NY Armonk
- Agent Steven F. McDaniel
- Main IPC: G06F9/46
- IPC: G06F9/46 ; G06F9/48 ; G06F9/50

Abstract:
A method for scheduling MapReduce tasks includes receiving a set of task statistics corresponding to task execution within a MapReduce job, estimating a completion time for a set of tasks to be executed to provide an estimated completion time, calculating a soft decision point based on a convergence of a workload distribution corresponding to a set of executed tasks, calculating a hard decision point based on the estimated completion time for the set of tasks to be executed, determining a selected decision point based on the soft decision point and the hard decision point, and scheduling upcoming tasks for execution based on the selected decision point. The method may also include estimating a map task completion time and estimating a shuffle operation completion time. A computer program product and computer system corresponding to the method are also disclosed.
Information query