公开(公告)号：US12229667B2

公开(公告)日：2025-02-18

申请号：US17209576

申请日：2021-03-23

Applicant: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY CO., LTD.

Inventor： Daxiang Dong ,  Wenhui Zhang ,  Zhihua Wu ,  Dianhai Yu ,  Yanjun Ma ,  Haifeng Wang

IPC: G06N3/08 ,  G06F9/50 ,  G06F18/214

Abstract: A method and an apparatus for generating a shared encoder are provided, which belongs to a field of computer technology and deep learning. The method includes: sending by a master node a shared encoder training instruction to child nodes, so that each child node obtains training samples based on a type of a target shared encoder included in the training instruction; sending an initial parameter set of the target shared encoder to be trained to each child node after obtaining a confirmation message returned by each child node; obtaining an updated parameter set of the target shared encoder returned by each child node; determining a target parameter set corresponding to the target shared encoder based on a first preset rule and the updated parameter set of the target shared encoder returned by each child node.

公开(公告)号：US20210326762A1

公开(公告)日：2021-10-21

申请号：US17351194

申请日：2021-06-17

Applicant: Beijing Baidu Netcom Science and Technology Co., Ltd.

Inventor： Zhihua Wu ,  Dianhai Yu ,  Xuefeng Yao ,  Wei Tang ,  Xinxuan Wu ,  Mo Cheng ,  Lin Ma ,  Yanjun Ma ,  Tian Wu ,  Haifeng Wang

IPC: G06N20/00

Abstract: The present disclosure discloses an apparatus and method for distributedly training a model, an electronic device, and a computer readable storage medium. The apparatus may include: a distributed reader, a distributed trainer and a distributed parameter server that are mutually independent. A reader in the distributed reader is configured to acquire a training sample, and load the acquired training sample to a corresponding trainer in the distributed trainer; the trainer in the distributed trainer is configured to perform model training based on the loaded training sample to obtain gradient information; and a parameter server in the distributed parameter server is configured to update a parameter of an initial model based on the gradient information of the distributed trainer to obtain a trained target model.