-
公开(公告)号：US20210374334A1
公开(公告)日：2021-12-02
申请号：US17117211
申请日：2020-12-10
Abstract: A method for training a language model, an electronic device and a readable storage medium, which relate to the field of natural language processing technologies in artificial intelligence, are disclosed. The method may include pre-training the language model using preset text language materials in a corpus; replacing at least one word in a sample text language material with a word mask respectively to obtain a sample text language material including at least one word mask; inputting the sample text language material including the at least one word mask into the language model, and outputting a context vector of each of the at least one word mask via the language model; determining a word vector corresponding to each word mask based on the context vector of the word mask and a word vector parameter matrix; and training the language model based on the word vector corresponding to each word mask.
-
2.发明申请公开(公告)号：US20210374343A1
公开(公告)日：2021-12-02
申请号：US17095955
申请日：2020-11-12
IPC: G06F40/279 , G06F40/205 , G06F16/9032 , G06K9/62 , G06N20/00
Abstract: A method and apparatus for obtaining word vectors based on a language model, a device and a storage medium are disclosed, which relates to the field of natural language processing technologies in artificial intelligence. An implementation includes inputting each of at least two first sample text language materials into the language model, and outputting a context vector of a first word mask in each first sample text language material via the language model; determining the word vector corresponding to each first word mask based on a first word vector parameter matrix, a second word vector parameter matrix and a fully connected matrix respectively; and training the language model and the fully connected matrix based on the word vectors corresponding to the first word masks in the at least two first sample text language materials, so as to obtain the word vectors.
-
公开(公告)号：US20210374352A1
公开(公告)日：2021-12-02
申请号：US16951702
申请日：2020-11-18
IPC: G06F40/30 , G06F40/166 , G06F40/279 , G06N20/00
Abstract: A method for training a language model based on various word vectors, a device and a medium, which relate to the field of natural language processing technologies in artificial intelligence, are disclosed. An implementation includes inputting a first sample text language material including a first word mask into the language model, and outputting a context vector of the first word mask via the language model; acquiring a first probability distribution matrix of the first word mask based on the context vector of the first word mask and a first word vector parameter matrix, and a second probability distribution matrix of the first word mask based on the context vector of the first word mask and a second word vector parameter matrix; and training the language model based on a word vector corresponding to the first word mask.
-
-
Search Results
3
records
(took 0.016 seconds)
