公开(公告)号：US20210390257A1

公开(公告)日：2021-12-16

申请号：US17116846

申请日：2020-12-09

Applicant: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY CO., LTD.

Inventor： Chao Pang ,  Shuohuan Wang ,  Yu Sun ,  Hua Wu ,  Haifeng Wang

IPC: G06F40/295 ,  G06F40/30 ,  G06F40/137 ,  G06N5/02

Abstract: A method, an apparatus, a device and a storage medium for learning a knowledge representation are provided. The method can include: sampling a sub-graph of a knowledge graph from a knowledge base; serializing the sub-graph of the knowledge graph to obtain a serialized text; and reading using a pre-trained language model the serialized text in an order in the sub-graph of the knowledge graph, to perform learning to obtain a knowledge representation of each word in the serialized text. The knowledge representation learning in this embodiment is performed for entity and relationship representation learning in the knowledge base.

公开(公告)号：US11687718B2

公开(公告)日：2023-06-27

申请号：US17116846

申请日：2020-12-09

Applicant: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY CO., LTD.

Inventor： Chao Pang ,  Shuohuan Wang ,  Yu Sun ,  Hua Wu ,  Haifeng Wang

IPC: G06F17/00 ,  G06F40/295 ,  G06F40/137 ,  G06F40/30

CPC classification number: G06F40/295 ,  G06F40/137 ,  G06F40/30

Abstract: A method, an apparatus, a device and a storage medium for learning a knowledge representation are provided. The method can include: sampling a sub-graph of a knowledge graph from a knowledge base; serializing the sub-graph of the knowledge graph to obtain a serialized text; and reading using a pre-trained language model the serialized text in an order in the sub-graph of the knowledge graph, to perform learning to obtain a knowledge representation of each word in the serialized text. The knowledge representation learning in this embodiment is performed for entity and relationship representation learning in the knowledge base.

公开(公告)号：US11995405B2

公开(公告)日：2024-05-28

申请号：US17348104

申请日：2021-06-15

Applicant: BEIJING BAIDU NETCOM SCIENCE AND TECHNOLOGY CO., LTD.

Inventor： Xuan Ouyang ,  Shuohuan Wang ,  Chao Pang ,  Yu Sun ,  Hao Tian ,  Hua Wu ,  Haifeng Wang

IPC: G06F40/30 ,  G06F40/58 ,  G06N20/00

CPC classification number: G06F40/30 ,  G06F40/58 ,  G06N20/00

Abstract: The present disclosure provides a multi-lingual model training method, apparatus, electronic device and readable storage medium and relates to the technical field of deep learning and natural language processing. A technical solution of the present disclosure when training the multi-lingual model is: obtaining training corpuses comprising a plurality of bilingual corpuses and a plurality of monolingual corpuses; training a multi-lingual model with a first training task by using the plurality of bilingual corpuses; training the multi-lingual model with a second training task by using the plurality of monolingual corpuses; and completing the training of the multi-lingual model in a case of determining that loss functions of the first training task and second training task converge. In the present disclosure, the multi-lingual model can be enabled to achieve semantic interaction between different languages and improve the accuracy of the multi-lingual model in learning the semantic representations of the multi-lingual model.