-
公开(公告)号：US10380161B2
公开(公告)日：2019-08-13
申请号：US15815686
申请日：2017-11-16
Applicant: salesforce.com, inc.
Inventor： Romain Paulus
Abstract: Disclosed RNN-implemented methods and systems for abstractive text summarization process input token embeddings of a document through an encoder that produces encoder hidden states; applies the decoder hidden state to encoder hidden states to produce encoder attention scores for encoder hidden states; generates encoder temporal scores for the encoder hidden states by exponentially normalizing a particular encoder hidden state's encoder attention score over its previous encoder attention scores; generates normalized encoder temporal scores by unity normalizing the temporal scores; produces the intra-temporal encoder attention vector; applies the decoder hidden state to each of previous decoder hidden states to produce decoder attention scores for each of the previous decoder hidden states; generates normalized decoder attention scores for previous decoder hidden states by exponentially normalizing each of the decoder attention scores; identifies previously predicted output tokens; produces the intra-decoder attention vector and processes the vector to emit a summary token.
-
公开(公告)号：US20190311002A1
公开(公告)日：2019-10-10
申请号：US16452339
申请日：2019-06-25
Applicant: salesforce.com, inc.
Inventor： Romain Paulus
Abstract: A system for text summarization includes an encoder for encoding input tokens of a document and a decoder for emitting summary tokens which summarize the document based on the encoded input tokens. At each iteration the decoder generates attention scores between a current hidden state of the decoder and previous hidden states of the decoder, generates a current decoder context from the attention scores and the previous hidden states of the decoder, and selects a next summary token based on the current decoder context and a current encoder context of the encoder. The attention scores penalize candidate summary tokens having high attention scores in previous iterations. In some embodiments, the attention scores include an attention score for each of the previous hidden states of the decoder. In some embodiments, the selection of the next summary token prevents emission of repeated summary phrases in a summary of the document.
-
公开(公告)号：US20160350653A1
公开(公告)日：2016-12-01
申请号：US15170884
申请日：2016-06-01
Applicant: salesforce.com, inc.
Inventor： Richard Socher , Ankit Kumar , Ozan Irsoy , Mohit Iyyer , Caiming Xiong , Stephen Merity , Romain Paulus
CPC classification number: G06N5/04 , G06N3/0445
Abstract: A novel unified neural network framework, the dynamic memory network, is disclosed. This unified framework reduces every task in natural language processing to a question answering problem over an input sequence. Inputs and questions are used to create and connect deep memory sequences. Answers are then generated based on dynamically retrieved memories.
Abstract translation: 公开了一种新颖的统一神经网络框架，动态存储网络。 这个统一框架将自然语言处理中的每个任务都减少到一个输入序列中的问题回答问题。 输入和问题用于创建和连接深层记忆序列。 然后基于动态检索的存储器生成答案。
-
公开(公告)号：US11003704B2
公开(公告)日：2021-05-11
申请号：US16696527
申请日：2019-11-26
Applicant: salesforce.com, inc.
Inventor： Romain Paulus
Abstract: A system for text summarization includes an encoder for encoding input tokens of a document and a decoder for emitting summary tokens which summarize the document based on the encoded input tokens. At each iteration the decoder generates attention scores between a current hidden state of the decoder and previous hidden states of the decoder, generates a current decoder context from the attention scores and the previous hidden states of the decoder, and selects a next summary token based on the current decoder context. The selection of the next summary token prevents emission of repeated summary phrases in a summary of the document.
-
公开(公告)号：US20170024645A1
公开(公告)日：2017-01-26
申请号：US15221532
申请日：2016-07-27
Applicant: salesforce.com, inc.
Inventor： Richard Socher , Ankit Kumar , Ozan Irsoy , Mohit Iyyer , Caiming Xiong , Stephen Merity , Romain Paulus
CPC classification number: G06N3/08 , G06F16/3329 , G06F16/3347 , G06N3/0427 , G06N3/0445
Abstract: A novel unified neural network framework, the dynamic memory network, is disclosed. This unified framework reduces every task in natural language processing to a question answering problem over an input sequence. Inputs and questions are used to create and connect deep memory sequences. Answers are then generated based on dynamically retrieved memories.
Abstract translation: 公开了一种新颖的统一神经网络框架，动态存储网络。 这个统一框架将自然语言处理中的每个任务都减少到一个输入序列中的问题回答问题。 输入和问题用于创建和连接深层记忆序列。 然后基于动态检索的存储器生成答案。
-
Patent Agency Ranking
公开(公告)号：US11113598B2
公开(公告)日：2021-09-07
申请号：US15221532
申请日：2016-07-27
Applicant: salesforce.com, inc.
Inventor： Richard Socher , Ankit Kumar , Ozan Irsoy , Mohit Iyyer , Caiming Xiong , Stephen Merity , Romain Paulus
IPC: G06N3/08 , G06N3/04 , G06F16/33 , G06F16/332
Abstract: A novel unified neural network framework, the dynamic memory network, is disclosed. This unified framework reduces every task in natural language processing to a question answering problem over an input sequence. Inputs and questions are used to create and connect deep memory sequences. Answers are then generated based on dynamically retrieved memories.
-
公开(公告)号：US10474709B2
公开(公告)日：2019-11-12
申请号：US15815686
申请日：2017-11-16
Applicant: salesforce.com, inc.
Inventor： Romain Paulus
Abstract: Disclosed RNN-implemented methods and systems for abstractive text summarization process input token embeddings of a document through an encoder that produces encoder hidden states; applies the decoder hidden state to encoder hidden states to produce encoder attention scores for encoder hidden states; generates encoder temporal scores for the encoder hidden states by exponentially normalizing a particular encoder hidden state's encoder attention score over its previous encoder attention scores; generates normalized encoder temporal scores by unity normalizing the temporal scores; produces the intra-temporal encoder attention vector; applies the decoder hidden state to each of previous decoder hidden states to produce decoder attention scores for each of the previous decoder hidden states; generates normalized decoder attention scores for previous decoder hidden states by exponentially normalizing each of the decoder attention scores; identifies previously predicted output tokens; produces the intra-decoder attention vector and processes the vector to emit a summary token.
-
公开(公告)号：US10909157B2
公开(公告)日：2021-02-02
申请号：US16051188
申请日：2018-07-31
Applicant: salesforce.com, inc.
Inventor： Romain Paulus , Wojciech Kryscinski , Caiming Xiong
Abstract: A system is disclosed for providing an abstractive summary of a source textual document. The system includes an encoder, a decoder, and a fusion layer. The encoder is capable of generating an encoding for the source textual document. The decoder is separated into a contextual model and a language model. The contextual model is capable of extracting words from the source textual document using the encoding. The language model is capable of generating vectors paraphrasing the source textual document based on pre-training with a training dataset. The fusion layer is capable of generating the abstractive summary of the source textual document from the extracted words and the generated vectors for paraphrasing. In some embodiments, the system utilizes a novelty metric to encourage the generation of novel phrases for inclusion in the abstractive summary.
-
公开(公告)号：US10521465B2
公开(公告)日：2019-12-31
申请号：US16452339
申请日：2019-06-25
Applicant: salesforce.com, inc.
Inventor： Romain Paulus
Abstract: A system for text summarization includes an encoder for encoding input tokens of a document and a decoder for emitting summary tokens which summarize the document based on the encoded input tokens. At each iteration the decoder generates attention scores between a current hidden state of the decoder and previous hidden states of the decoder, generates a current decoder context from the attention scores and the previous hidden states of the decoder, and selects a next summary token based on the current decoder context and a current encoder context of the encoder. The attention scores penalize candidate summary tokens having high attention scores in previous iterations. In some embodiments, the attention scores include an attention score for each of the previous hidden states of the decoder. In some embodiments, the selection of the next summary token prevents emission of repeated summary phrases in a summary of the document.
-
公开(公告)号：US20180300400A1
公开(公告)日：2018-10-18
申请号：US15815686
申请日：2017-11-16
Applicant: salesforce.com, inc.
Inventor： Romain Paulus
Abstract: Disclosed RNN-implemented methods and systems for abstractive text summarization process input token embeddings of a document through an encoder that produces encoder hidden states; applies the decoder hidden state to encoder hidden states to produce encoder attention scores for encoder hidden states; generates encoder temporal scores for the encoder hidden states by exponentially normalizing a particular encoder hidden state's encoder attention score over its previous encoder attention scores; generates normalized encoder temporal scores by unity normalizing the temporal scores; produces the intra-temporal encoder attention vector; applies the decoder hidden state to each of previous decoder hidden states to produce decoder attention scores for each of the previous decoder hidden states; generates normalized decoder attention scores for previous decoder hidden states by exponentially normalizing each of the decoder attention scores; identifies previously predicted output tokens; produces the intra-decoder attention vector and processes the vector to emit a summary token.
-
-
-
-
-
-
-
-
-
Search Results
10
records
(took 0.019 seconds)
