公开(公告)号：US11709894B2

公开(公告)日：2023-07-25

申请号：US16352678

申请日：2019-03-13

Applicant: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY CO., LTD.

Inventor： Yuncong Zhang ,  Xiang Wen ,  Hua Chai ,  Cong Wang ,  Guanyin Zhu

IPC: G06F16/903 ,  G06F16/2455 ,  G06F16/2452 ,  G06F40/205 ,  G06F16/182 ,  G06F16/9032 ,  G06F16/9035 ,  G06F16/901 ,  G06F9/50

CPC classification number: G06F16/90335 ,  G06F16/182 ,  G06F16/24524 ,  G06F16/24556 ,  G06F16/9027 ,  G06F16/9035 ,  G06F16/90332 ,  G06F40/205 ,  G06F9/5072

Abstract: The present disclosure discloses a task processing method and a distributed computing framework. A specific embodiment of the method includes: parsing an expression corresponding to a distributed computing task, and constructing task description information corresponding to the distributed computing task, the task description information being used to describe a corresponding relationship between an operator and a distributed dataset, and the operator acting on at least one of the distributed dataset or distributed datasets obtained by grouping the distributed dataset; determining, based on the task description information, a distributed dataset the operator acting on; and performing distributed computing on the distributed dataset the operator acting on using the operator. In the distributed computing, the acting scope and nesting relationship of the operator is described by constructing a topology.

公开(公告)号：US11132363B2

公开(公告)日：2021-09-28

申请号：US16352576

申请日：2019-03-13

Applicant: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY CO., LTD.

Inventor： Jianwei Zhang ,  Yuncong Zhang ,  Cong Wang ,  Yao Xu ,  Chunyang Wen ,  Xin Huang ,  Zhan Song ,  Guanyin Zhu

IPC: G06F16/24 ,  G06F16/2453 ,  G06F9/50 ,  G06F40/205 ,  G06F16/242 ,  G06F16/182 ,  G06F16/22

Abstract: A distributed computing framework and a distributed computing method are provided. A specific embodiment of the distributed computing framework includes: a parsing unit, configured to parse an expression of a distributed computing task, and determine an operator and a field corresponding to the operator; and an operator unit, configured to provide the operator, input parameters of the operator including: the field and a field-type distributed dataset. The type of parameters received and returned by any operator may be the field-type distributed dataset, and any operator may operate on the data corresponding to the field in the field-type distributed dataset. Therefore, any operator needs to be implemented once to realize the reuse of the operator. The distributed computing task is expressed in a simple expression, which simplifies the complexity of writing a distributed computing program with the distributed computing framework used by the user.