公开(公告)号：US12026532B2

公开(公告)日：2024-07-02

申请号：US18229450

申请日：2023-08-02

Applicant: Google LLC

Inventor： Craig D. Chambers ,  Ashish Raniwala ,  Frances J. Perry ,  Stephen R. Adams ,  Robert R. Henry ,  Robert Bradshaw ,  Nathan Weizenbaum

IPC: G06F9/46 ,  G06F8/30 ,  G06F8/34 ,  G06F8/41 ,  G06F9/38 ,  G06F9/44 ,  G06F9/445 ,  G06F9/455 ,  G06F9/48 ,  G06F16/25 ,  G06F21/57 ,  G06F21/62 ,  G06F9/30 ,  G06F9/448 ,  G06F16/2453

CPC classification number: G06F9/45504 ,  G06F8/314 ,  G06F8/34 ,  G06F8/433 ,  G06F9/38 ,  G06F9/3851 ,  G06F9/3885 ,  G06F9/44 ,  G06F9/445 ,  G06F9/45533 ,  G06F9/4843 ,  G06F21/577 ,  G06F21/62 ,  G06F21/6218 ,  G06F9/30 ,  G06F9/4494 ,  G06F16/24532 ,  G06F16/24547 ,  G06F2221/034

Abstract: A data parallel pipeline may specify multiple parallel data objects that contain multiple elements and multiple parallel operations that operate on the parallel data objects. Based on the data parallel pipeline, a dataflow graph of deferred parallel data objects and deferred parallel operations corresponding to the data parallel pipeline may be generated and one or more graph transformations may be applied to the dataflow graph to generate a revised dataflow graph that includes one or more of the deferred parallel data objects and deferred, combined parallel data operations. The deferred, combined parallel operations may be executed to produce materialized parallel data objects corresponding to the deferred parallel data objects.

公开(公告)号：US20230376332A1

公开(公告)日：2023-11-23

申请号：US18229450

申请日：2023-08-02

Applicant: Google LLC

Inventor： Craig D. Chambers ,  Ashish Raniwala ,  Frances J. Perry ,  Stephen R. Adams ,  Robert R. Henry ,  Robert Bradshaw ,  Nathan Weizenbaum

IPC: G06F9/455 ,  G06F21/62 ,  G06F9/38 ,  G06F8/41 ,  G06F9/48 ,  G06F8/30 ,  G06F8/34 ,  G06F9/44 ,  G06F9/445 ,  G06F21/57

CPC classification number: G06F9/45504 ,  G06F21/62 ,  G06F9/38 ,  G06F9/3851 ,  G06F9/3885 ,  G06F8/433 ,  G06F9/4843 ,  G06F8/314 ,  G06F8/34 ,  G06F9/44 ,  G06F9/45533 ,  G06F9/445 ,  G06F21/577 ,  G06F21/6218 ,  G06F16/24532

Abstract: A data parallel pipeline may specify multiple parallel data objects that contain multiple elements and multiple parallel operations that operate on the parallel data objects. Based on the data parallel pipeline, a dataflow graph of deferred parallel data objects and deferred parallel operations corresponding to the data parallel pipeline may be generated and one or more graph transformations may be applied to the dataflow graph to generate a revised dataflow graph that includes one or more of the deferred parallel data objects and deferred, combined parallel data operations. The deferred, combined parallel operations may be executed to produce materialized parallel data objects corresponding to the deferred parallel data objects.

公开(公告)号：US10732928B1

公开(公告)日：2020-08-04

申请号：US16003997

申请日：2018-06-08

Applicant: Google LLC

Inventor： Tyler Akidau ,  Robert Bradshaw ,  Ben Chambers ,  Craig Chambers ,  Reuven Lax ,  Daniel Mills ,  Frances Perry

IPC: G06F7/00 ,  H04L12/841 ,  H04L12/70

Abstract: A method includes receiving data corresponding one of streaming data or batch data and a content of the received data for computation. The method also includes determining an event time of the data for slicing the data, determining a processing time to output results of the received data, and emitting at least a portion of the results of the received data based on the processing time and the event time.

公开(公告)号：US10025620B2

公开(公告)日：2018-07-17

申请号：US14989867

申请日：2016-01-07

Applicant: Google LLC

Inventor： Robert Bradshaw ,  Craig D. Chambers ,  Ezra Cooper ,  Ashish Raniwala ,  Frances J. Perry

IPC: G06F9/455 ,  G06F9/45 ,  G06F17/30 ,  G06F9/46 ,  G06F9/38 ,  G06F8/41 ,  G06F9/50 ,  G06F11/30

Abstract: One example method includes identifying synchronous code including instructions specifying a computing operation to be performed on a set of data; transforming the synchronous code into a pipeline application including one or more pipeline objects; identifying a first input data set on which to execute the pipeline application; executing the pipeline application on a first input data set to produce a first output data set; after executing the pipeline application on the first input data set, identifying a second input data set on which to execute the pipeline application; determining a set of differences between the first input data set and second input data set; and executing the pipeline application on the set of differences to produce a second output data set.

公开(公告)号：US11755351B2

公开(公告)日：2023-09-12

申请号：US17834256

申请日：2022-06-07

Applicant: Google LLC

Inventor： Craig D. Chambers ,  Ashish Raniwala ,  Frances J. Perry ,  Stephen R. Adams ,  Robert R. Henry ,  Robert Bradshaw ,  Nathan Weizenbaum

IPC: G06F9/38 ,  G06F16/2453 ,  G06F21/57 ,  G06F9/445 ,  G06F9/455 ,  G06F21/62 ,  G06F8/41 ,  G06F9/48 ,  G06F8/30 ,  G06F8/34 ,  G06F9/44 ,  G06F9/448 ,  G06F9/30

CPC classification number: G06F9/45504 ,  G06F8/314 ,  G06F8/34 ,  G06F8/433 ,  G06F9/38 ,  G06F9/3851 ,  G06F9/3885 ,  G06F9/44 ,  G06F9/445 ,  G06F9/45533 ,  G06F9/4843 ,  G06F21/577 ,  G06F21/62 ,  G06F21/6218 ,  G06F9/30 ,  G06F9/4494 ,  G06F16/24532 ,  G06F16/24547 ,  G06F2221/034

Abstract: A data parallel pipeline may specify multiple parallel data objects that contain multiple elements and multiple parallel operations that operate on the parallel data objects. Based on the data parallel pipeline, a dataflow graph of deferred parallel data objects and deferred parallel operations corresponding to the data parallel pipeline may be generated and one or more graph transformations may be applied to the dataflow graph to generate a revised dataflow graph that includes one or more of the deferred parallel data objects and deferred, combined parallel data operations. The deferred, combined parallel operations may be executed to produce materialized parallel data objects corresponding to the deferred parallel data objects.

公开(公告)号：US20200225991A1

公开(公告)日：2020-07-16

申请号：US16781467

申请日：2020-02-04

Applicant: Google LLC

Inventor： Robert Bradshaw ,  Rafael de Jesús Fernández Moctezuma ,  Daniel Mills ,  Samuel Green McVeety ,  Samuel Carl Whittle ,  Andrei Maksimenka ,  Cosmin Ionel Arad ,  Mark Brian Shields ,  Harris Samuel Nover ,  Manuel Alfred Fahndrich ,  Jeffrey Paul Gardner ,  Mikhail Shmulyan ,  Reuven Lax ,  Ahmet Altay ,  Craig D. Chambers

IPC: G06F9/50

Abstract: A job that receives as input a data stream is executed. For the job, it is iteratively determined a backlog growth over a first period of time; a backlog volume; and whether to adjust a quantity of processing resources. For each iteration for which the quantity of processing resources allocated to the job are determined to be adjusted, adjusting the quantity of processing resources allocated to the job. For each iteration for which the quantity of processing resources allocated to the job are determined not to be adjusted, maintaining the quantity of processing resources allocated to the job.

公开(公告)号：US20240338235A1

公开(公告)日：2024-10-10

申请号：US18748889

申请日：2024-06-20

Applicant: Google LLC

Inventor： Craig D. Chambers ,  Ashish Raniwala ,  Frances J. Perry ,  Stephen R. Adams ,  Robert R. Henry ,  Robert Bradshaw ,  Nathan Weizenbaum

IPC: G06F9/455 ,  G06F8/30 ,  G06F8/34 ,  G06F8/41 ,  G06F9/30 ,  G06F9/38 ,  G06F9/44 ,  G06F9/445 ,  G06F9/448 ,  G06F9/48 ,  G06F16/2453 ,  G06F21/57 ,  G06F21/62

CPC classification number: G06F9/45504 ,  G06F8/314 ,  G06F8/34 ,  G06F8/433 ,  G06F9/38 ,  G06F9/3851 ,  G06F9/3885 ,  G06F9/44 ,  G06F9/445 ,  G06F9/45533 ,  G06F9/4843 ,  G06F21/577 ,  G06F21/62 ,  G06F21/6218 ,  G06F9/30 ,  G06F9/4494 ,  G06F16/24532 ,  G06F16/24547 ,  G06F2221/034

Abstract: A data parallel pipeline may specify multiple parallel data objects that contain multiple elements and multiple parallel operations that operate on the parallel data objects. Based on the data parallel pipeline, a dataflow graph of deferred parallel data objects and deferred parallel operations corresponding to the data parallel pipeline may be generated and one or more graph transformations may be applied to the dataflow graph to generate a revised dataflow graph that includes one or more of the deferred parallel data objects and deferred, combined parallel data operations. The deferred, combined parallel operations may be executed to produce materialized parallel data objects corresponding to the deferred parallel data objects.

公开(公告)号：US10558501B2

公开(公告)日：2020-02-11

申请号：US15395225

申请日：2016-12-30

Applicant: Google LLC

Inventor： Robert Bradshaw ,  Rafael de Jesús Fernández Moctezuma ,  Daniel Mills ,  Samuel Green McVeety ,  Samuel Carl Whittle ,  Andrei Maksimenka ,  Cosmin Ionel Arad ,  Mark Brian Shields ,  Harris Samuel Nover ,  Manuel Alfred Fahndrich ,  Jeffrey Paul Gardner ,  Mikhail Shmulyan ,  Reuven Lax ,  Ahmet Altay ,  Craig D. Chambers

IPC: G06F9/50 ,  H04L29/08

Abstract: A job that receives as input a data stream is executed. For the job, it is iteratively determined a backlog growth over a first period of time; a backlog volume; and whether to adjust a quantity of processing resources. For each iteration for which the quantity of processing resources allocated to the job are determined to be adjusted, adjusting the quantity of processing resources allocated to the job. For each iteration for which the quantity of processing resources allocated to the job are determined not to be adjusted, maintaining the quantity of processing resources allocated to the job.

公开(公告)号：US10133592B2

公开(公告)日：2018-11-20

申请号：US15483044

申请日：2017-04-10

Applicant: Google LLC

Inventor： Craig D. Chambers ,  Ashish Raniwala ,  Frances J. Perry ,  Stephen R. Adams ,  Robert R. Henry ,  Robert Bradshaw ,  Nathan Weizenbaum

IPC: G06F9/44 ,  G06F9/45 ,  G06F17/30 ,  G06F9/455 ,  G06F21/62 ,  G06F9/38 ,  G06F8/41 ,  G06F9/48 ,  G06F8/30 ,  G06F8/34 ,  G06F9/445 ,  G06F21/57 ,  G06F9/30 ,  G06F9/448

Abstract: A data parallel pipeline may specify multiple parallel data objects that contain multiple elements and multiple parallel operations that operate on the parallel data objects. Based on the data parallel pipeline, a dataflow graph of deferred parallel data objects and deferred parallel operations corresponding to the data parallel pipeline may be generated and one or more graph transformations may be applied to the dataflow graph to generate a revised dataflow graph that includes one or more of the deferred parallel data objects and deferred, combined parallel data operations. The deferred, combined parallel operations may be executed to produce materialized parallel data objects corresponding to the deferred parallel data objects.

公开(公告)号：US20180314545A1

公开(公告)日：2018-11-01

申请号：US16030183

申请日：2018-07-09

Applicant: Google LLC

Inventor： Robert Bradshaw ,  Craig D. Chambers ,  Ezra Cooper ,  Ashish Raniwala ,  Frances J. Perry

IPC: G06F9/46 ,  G06F9/50 ,  G06F8/41 ,  G06F9/38 ,  G06F9/455

CPC classification number: G06F9/466 ,  G06F8/453 ,  G06F9/3867 ,  G06F9/3885 ,  G06F9/455 ,  G06F9/5066 ,  G06F11/3006 ,  G06F17/30463 ,  G06F17/30569

Abstract: One example method includes identifying synchronous code including instructions specifying a computing operation to be performed on a set of data; transforming the synchronous code into a pipeline application including one or more pipeline objects; identifying a first input data set on which to execute the pipeline application; executing the pipeline application on a first input data set to produce a first output data set; after executing the pipeline application on the first input data set, identifying a second input data set on which to execute the pipeline application; determining a set of differences between the first input data set and second input data set; and executing the pipeline application on the set of differences to produce a second output data set.