-
公开(公告)号：US20210374334A1
公开(公告)日：2021-12-02
申请号：US17117211
申请日：2020-12-10
Abstract: A method for training a language model, an electronic device and a readable storage medium, which relate to the field of natural language processing technologies in artificial intelligence, are disclosed. The method may include pre-training the language model using preset text language materials in a corpus; replacing at least one word in a sample text language material with a word mask respectively to obtain a sample text language material including at least one word mask; inputting the sample text language material including the at least one word mask into the language model, and outputting a context vector of each of the at least one word mask via the language model; determining a word vector corresponding to each word mask based on the context vector of the word mask and a word vector parameter matrix; and training the language model based on the word vector corresponding to each word mask.
-
公开(公告)号：US20210232775A1
公开(公告)日：2021-07-29
申请号：US17031569
申请日：2020-09-24
Inventor： Han ZHANG , Dongling XIAO , Yukun LI , Yu SUN , Hao TIAN , Hua WU , Haifeng WANG
IPC: G06F40/56
Abstract: The present disclosure proposes a language generation method and apparatus. The method includes: performing encoding processing on an input sequence by using a preset encoder to generate a hidden state vector corresponding to the input sequence; in response to a granularity category of a second target segment being a phrase, decoding a first target segment vector, the hidden state vector, and a position vector corresponding to the second target segment by using N decoders to generate N second target segments; determining a loss value based on differences between respective N second target segments and a second target annotated segment; and performing parameter updating on the preset encoder, a preset classifier, and the N decoders based on the loss value to generate an updated language generation model for performing language generation.
-
Patent Agency Ranking
3.发明申请公开(公告)号：US20210374343A1
公开(公告)日：2021-12-02
申请号：US17095955
申请日：2020-11-12
IPC: G06F40/279 , G06F40/205 , G06F16/9032 , G06K9/62 , G06N20/00
Abstract: A method and apparatus for obtaining word vectors based on a language model, a device and a storage medium are disclosed, which relates to the field of natural language processing technologies in artificial intelligence. An implementation includes inputting each of at least two first sample text language materials into the language model, and outputting a context vector of a first word mask in each first sample text language material via the language model; determining the word vector corresponding to each first word mask based on a first word vector parameter matrix, a second word vector parameter matrix and a fully connected matrix respectively; and training the language model and the fully connected matrix based on the word vectors corresponding to the first word masks in the at least two first sample text language materials, so as to obtain the word vectors.
-
公开(公告)号：US20210374352A1
公开(公告)日：2021-12-02
申请号：US16951702
申请日：2020-11-18
IPC: G06F40/30 , G06F40/166 , G06F40/279 , G06N20/00
Abstract: A method for training a language model based on various word vectors, a device and a medium, which relate to the field of natural language processing technologies in artificial intelligence, are disclosed. An implementation includes inputting a first sample text language material including a first word mask into the language model, and outputting a context vector of the first word mask via the language model; acquiring a first probability distribution matrix of the first word mask based on the context vector of the first word mask and a first word vector parameter matrix, and a second probability distribution matrix of the first word mask based on the context vector of the first word mask and a second word vector parameter matrix; and training the language model based on a word vector corresponding to the first word mask.
-
公开(公告)号：US20210182498A1
公开(公告)日：2021-06-17
申请号：US16885358
申请日：2020-05-28
Inventor： Yu SUN , Haifeng WANG , Shuohuan WANG , Yukun LI , Shikun FENG , Hao TIAN , Hua WU
Abstract: The present disclosure provides a method, apparatus, electronic device and storage medium for processing a semantic representation model, and relates to the field of artificial intelligence technologies. A specific implementation solution is: collecting a training corpus set including a plurality of training corpuses; training the semantic representation model using the training corpus set based on at least one of lexicon, grammar and semantics. In the present disclosure, by building the unsupervised or weakly-supervised training task at three different levels, namely, lexicon, grammar and semantics, the semantic representation model is enabled to learn knowledge at levels of lexicon, grammar and semantics from massive data, enhance the capability of universal semantic representation and improve the processing effect of the NLP task.
-
公开(公告)号：US20190065506A1
公开(公告)日：2019-02-28
申请号：US16054842
申请日：2018-08-03
Inventor： Yukun LI , Yi LIU , Yu SUN , Dianhai YU
Abstract: Embodiments of the present disclosure disclose a search method and apparatus based on artificial intelligence. A specific implementation of the method comprises: acquiring at least one candidate document related to a query sentence; determining a query word vector sequence corresponding to a segmented word sequence of the query sentence, and determining a candidate document word vector sequence corresponding to a segmented word sequence of each candidate document in the at least one candidate document; performing a similarity calculation for each candidate document in the at least one candidate document; selecting, in a descending order of similarities between the candidate document and the query sentence, a preset number of candidate documents from the at least one candidate document as a search result.
-
公开(公告)号：US20180329886A1
公开(公告)日：2018-11-15
申请号：US15900176
申请日：2018-02-20
Inventor： Yukun LI , Yi LIU , Yu SUN , Dianhai YU
CPC classification number: G06F17/2785 , G06F17/277 , G06N3/0454 , G06N3/0481 , G06N3/08
Abstract: An artificial intelligence based method and apparatus for generating information are disclosed. The method in an embodiment includes: segmenting a to-be-processed text into characters to obtain a character sequence; determining a character vector for each character in the character sequence to generate a character vector sequence; generating a plurality of character vector subsequences by segmenting the character vector sequence based on a preset vocabulary; for each generated character vector subsequence, determining a sum of character vectors composing the character vector subsequence as a target vector, and inputting the target vector into a pre-trained first neural network to obtain a word vector corresponding to the each character vector subsequence, the first neural network used to characterize a correspondence between the target vector and the word vector; and analyzing the to-be-processed text based on the obtained word vector to generate an analysis result. This embodiment improves the adaptability of text processing.
-
-
-
-
-
-
Search Results
7
records
(took 0.024 seconds)
