公开(公告)号：US20210209417A1

公开(公告)日：2021-07-08

申请号：US17209576

申请日：2021-03-23

Applicant: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY CO., LTD.

Inventor： Daxiang DONG ,  Wenhui ZHANG ,  Zhihua WU ,  Dianhai YU ,  Yanjun MA ,  Haifeng WANG

IPC: G06K9/62 ,  G06N3/08 ,  G06F9/50

Abstract: A method and an apparatus for generating a shared encoder are provided, which belongs to a field of computer technology and deep learning. The method includes: sending by a master node a shared encoder training instruction to child nodes, so that each child node obtains training samples based on a type of a target shared encoder included in the training instruction; sending an initial parameter set of the target shared encoder to be trained to each child node after obtaining a confirmation message returned by each child node; obtaining an updated parameter set of the target shared encoder returned by each child node; determining a target parameter set corresponding to the target shared encoder based on a first preset rule and the updated parameter set of the target shared encoder returned by each child node.

公开(公告)号：US20210357814A1

公开(公告)日：2021-11-18

申请号：US17362674

申请日：2021-06-29

Applicant: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY CO., LTD.

Inventor： Xinxuan WU ,  Xuefeng YAO ,  Dianhai YU ,  Zhihua WU ,  Yanjun MA ,  Tian WU ,  Haifeng WANG

IPC: G06N20/00

Abstract: The present disclosure provides a method and apparatus for distributed training a model, an electronic device, and a computer readable storage medium. The method may include: performing, for each batch of training samples acquired by a distributed first trainer, model training through a distributed second trainer to obtain gradient information; updating a target parameter in a distributed built-in parameter server according to the gradient information; and performing, in response to determining that training for a preset number of training samples is completed, a parameter exchange between the distributed built-in parameter server and a distributed parameter server through the distributed first trainer to perform a parameter update on the initial model until training for the initial model is completed.

公开(公告)号：US20210374542A1

公开(公告)日：2021-12-02

申请号：US17444687

申请日：2021-08-09

Applicant: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY CO., LTD.

Inventor： Wenhui ZHANG ,  Dianhai YU ,  Zhihua WU

IPC: G06N3/08 ,  G06F8/65

Abstract: The invention discloses a method and an apparatus for updating parameters of a multi-task model. The method includes: obtaining a training sample set, in which the training sample set comprises a plurality of samples and a task to which each sample belongs; putting each sample into a corresponding sample queue sequentially according to the task to which each sample belongs; training a shared network layer in the multi-task model and a target sub-network layer of tasks associated with the sample queue with samples in the sample queue in case that the number of the samples in the sample queue reaches a training data requirement, so as to generate a model parameter update gradient corresponding to the tasks associated with the sample queue; and updating parameters of the shared network layer and the target sub-network layer in a parameter server according to the model parameter update gradient.