Abstract:
Techniques are described for scheduling data access jobs based on a job dependency analysis. A requested primary data access job is analyzed to determine one or more preliminary data access jobs on which it depends, and an execution duration of each data access job is predicted based on historical data or other factors. A time-sensitive subset of the preliminary data access jobs is determined as the subset of those serially dependent preliminary data access jobs for which there is a minimum time difference between the total predicted execution duration and a requested target completion time. Data access jobs are scheduled with priority given to those preliminary data access jobs in the time-sensitive subset, to enable the primary data access jobs to be completed by the requested target completion times.
Abstract:
Techniques are described for scheduling data access jobs based on a job dependency analysis. Preliminary data access jobs subsequently accessed by a primary data access job are identified. Execution durations of the primary data access job and the preliminary data access jobs are determined based on historical data. A total execution duration of subsets of serially dependent preliminary data access jobs and predicted execution duration of primary data access job are determined. A time-sensitive subset is identified and start times for the data access job are scheduled. The results of the preliminary data access jobs are provided to the primary data access job prior to the start time of the primary data access job so the primary data access job completes by a target completion duration.