公开(公告)号：US20180268298A1

公开(公告)日：2018-09-20

申请号：US15853570

申请日：2017-12-22

Applicant: salesforce.com, inc.

Inventor： Alexander Rosenberg Johansen ,  Bryan McCann ,  James Bradbury ,  Richard Socher

IPC: G06N3/08 ,  G06N3/04 ,  G06K9/62 ,  G06F17/24 ,  G06F17/27 ,  G06N5/04 ,  G06F15/18

CPC classification number: G06N3/0454 ,  G06F15/76 ,  G06F17/241 ,  G06F17/2785 ,  G06K9/6262 ,  G06K9/6267 ,  G06K9/6268 ,  G06K9/6271 ,  G06N3/0445 ,  G06N3/0472 ,  G06N3/0481 ,  G06N3/084 ,  G06N5/04 ,  G06N20/00

Abstract: The technology disclosed proposes using a combination of computationally cheap, less-accurate bag of words (BoW) model and computationally expensive, more-accurate long short-term memory (LSTM) model to perform natural processing tasks such as sentiment analysis. The use of cheap, less-accurate BoW model is referred to herein as “skimming”. The use of expensive, more-accurate LSTM model is referred to herein as “reading”. The technology disclosed presents a probability-based guider (PBG). PBG combines the use of BoW model and the LSTM model. PBG uses a probability thresholding strategy to determine, based on the results of the BoW model, whether to invoke the LSTM model for reliably classifying a sentence as positive or negative. The technology disclosed also presents a deep neural network-based decision network (DDN) that is trained to learn the relationship between the BoW model and the LSTM model and to invoke only one of the two models.

公开(公告)号：US12235790B2

公开(公告)日：2025-02-25

申请号：US17670368

申请日：2022-02-11

Applicant: salesforce.com, inc.

Inventor： Alexander Rosenberg Johansen ,  Bryan McCann ,  James Bradbury ,  Richard Socher

IPC: G06N20/00 ,  G06F15/76 ,  G06F18/21 ,  G06F18/24 ,  G06F18/241 ,  G06F18/2413 ,  G06F40/169 ,  G06F40/30 ,  G06N3/04 ,  G06N3/044 ,  G06N3/045 ,  G06N3/047 ,  G06N3/048 ,  G06N3/08 ,  G06N3/084 ,  G06N5/04 ,  G06V10/764 ,  G06V10/776 ,  G06V10/82

Abstract: The technology disclosed proposes using a combination of computationally cheap, less-accurate bag of words (BoW) model and computationally expensive, more-accurate long short-term memory (LSTM) model to perform natural processing tasks such as sentiment analysis. The use of cheap, less-accurate BoW model is referred to herein as “skimming”. The use of expensive, more-accurate LSTM model is referred to herein as “reading”. The technology disclosed presents a probability-based guider (PBG). PBG combines the use of BoW model and the LSTM model. PBG uses a probability thresholding strategy to determine, based on the results of the BoW model, whether to invoke the LSTM model for reliably classifying a sentence as positive or negative. The technology disclosed also presents a deep neural network-based decision network (DDN) that is trained to learn the relationship between the BoW model and the LSTM model and to invoke only one of the two models.

公开(公告)号：US11922303B2

公开(公告)日：2024-03-05

申请号：US16877339

申请日：2020-05-18

Applicant: Salesforce.com, Inc.

Inventor： Wenhao Liu ,  Ka Chun Au ,  Shashank Harinath ,  Bryan McCann ,  Govardana Sachithanandam Ramachandran ,  Alexis Roos ,  Caiming Xiong

IPC: G06N3/00 ,  G06F40/40 ,  G06N3/045 ,  G06N3/08

CPC classification number: G06N3/08 ,  G06F40/40 ,  G06N3/045

Abstract: Embodiments described herein provides a training mechanism that transfers the knowledge from a trained BERT model into a much smaller model to approximate the behavior of BERT. Specifically, the BERT model may be treated as a teacher model, and a much smaller student model may be trained using the same inputs to the teacher model and the output from the teacher model. In this way, the student model can be trained within a much shorter time than the BERT teacher model, but with comparable performance with BERT.

公开(公告)号：US20220171943A1

公开(公告)日：2022-06-02

申请号：US17673709

申请日：2022-02-16

Applicant: salesforce.com, inc.

Inventor： Nitish Shirish Keskar ,  Bryan McCann ,  Richard Socher ,  Caiming Xiong

IPC: G06F40/30 ,  G06F40/284

Abstract: Systems and methods for unifying question answering and text classification via span extraction include a preprocessor for preparing a source text and an auxiliary text based on a task type of a natural language processing task, an encoder for receiving the source text and the auxiliary text from the preprocessor and generating an encoded representation of a combination of the source text and the auxiliary text, and a span-extractive decoder for receiving the encoded representation and identifying a span of text within the source text that is a result of the NLP task. The task type is one of entailment, classification, or regression. In some embodiments, the source text includes one or more of text received as input when the task type is entailment, a list of classifications when the task type is entailment or classification, or a list of similarity options when the task type is regression.

公开(公告)号：US20220067284A1

公开(公告)日：2022-03-03

申请号：US17125468

申请日：2020-12-17

Applicant: salesforce.com, inc.

Inventor： Junxian He ,  Wojciech Kryscinski ,  Bryan McCann

IPC: G06F40/284 ,  G06N7/00

Abstract: Embodiments described herein provide a flexible controllable summarization system that allows users to control the generation of summaries without manually editing or writing the summary, e.g., without the user actually adding or deleting certain information under various granularity. Specifically, the summarization system performs controllable summarization through keywords manipulation. A neural network model is learned to generate summaries conditioned on both the keywords and source document so that at test time a user can interact with the neural network model through a keyword interface, potentially enabling multi-factor control.

公开(公告)号：US11250311B2

公开(公告)日：2022-02-15

申请号：US15853570

申请日：2017-12-22

Applicant: salesforce.com, inc.

Inventor： Alexander Rosenberg Johansen ,  Bryan McCann ,  James Bradbury ,  Richard Socher

IPC: G06N3/04 ,  G06K9/62 ,  G06N20/00 ,  G06F15/76 ,  G06F40/30 ,  G06F40/16 ,  G06N3/08 ,  G06N5/04 ,  G06F40/169

Abstract: The technology disclosed proposes using a combination of computationally cheap, less-accurate bag of words (BoW) model and computationally expensive, more-accurate long short-term memory (LSTM) model to perform natural processing tasks such as sentiment analysis. The use of cheap, less-accurate BoW model is referred to herein as “skimming”. The use of expensive, more-accurate LSTM model is referred to herein as “reading”. The technology disclosed presents a probability-based guider (PBG). PBG combines the use of BoW model and the LSTM model. PBG uses a probability thresholding strategy to determine, based on the results of the BoW model, whether to invoke the LSTM model for reliably classifying a sentence as positive or negative. The technology disclosed also presents a deep neural network-based decision network (DDN) that is trained to learn the relationship between the BoW model and the LSTM model and to invoke only one of the two models.

公开(公告)号：US11003867B2

公开(公告)日：2021-05-11

申请号：US16399429

申请日：2019-04-30

Applicant: salesforce.com, inc.

Inventor： Jasdeep Singh ,  Nitish Shirish Keskar ,  Bryan McCann

IPC: G06F40/00 ,  G06F40/58 ,  G06N20/00 ,  G06N3/08 ,  G06F40/51

Abstract: Approaches for cross-lingual regularization for multilingual generalization include a method for training a natural language processing (NLP) deep learning module. The method includes accessing a first dataset having a first training data entry, the first training data entry including one or more natural language input text strings in a first language; translating at least one of the one or more natural language input text strings of the first training data entry from the first language to a second language; creating a second training data entry by starting with the first training data entry and substituting the at least one of the natural language input text strings in the first language with the translation of the at least one of the natural language input text strings in the second language; adding the second training data entry to a second dataset; and training the deep learning module using the second dataset.

公开(公告)号：US20210073459A1

公开(公告)日：2021-03-11

申请号：US17027130

申请日：2020-09-21

Applicant: salesforce.com, inc.

Inventor： Bryan McCann ,  Caiming Xiong ,  Richard Socher

IPC: G06F40/126 ,  G06N3/08 ,  G06N3/04 ,  G06F40/30 ,  G06F40/47 ,  G06F40/205 ,  G06F40/289

Abstract: A system is provided for natural language processing. In some embodiments, the system includes an encoder for generating context-specific word vectors for at least one input sequence of words. The encoder is pre-trained using training data for performing a first natural language processing task. A neural network performs a second natural language processing task on the at least one input sequence of words using the context-specific word vectors. The first natural language process task is different from the second natural language processing task and the neural network is separately trained from the encoder. In some embodiments, the first natural processing task can be machine translation, and the second natural processing task can be one of sentiment analysis, question classification, entailment classification, and question answering

公开(公告)号：US20190251168A1

公开(公告)日：2019-08-15

申请号：US15974118

申请日：2018-05-08

Applicant: salesforce.com, inc.

Inventor： Bryan McCann ,  Nitish Shirish Keskar ,  Caiming Xiong ,  Richard Socher

IPC: G06F17/27 ,  G06N3/08

Abstract: Approaches for multitask learning as question answering include an input layer for encoding a context and a question, a self-attention based transformer including an encoder and a decoder, a first bi-directional long-term short-term memory (biLSTM) for further encoding an output of the encoder, a long-term short-term memory (LSTM) for generating a context-adjusted hidden state from the output of the decoder and a hidden state, an attention network for generating first attention weights based on an output of the first biLSTM and an output of the LSTM, a vocabulary layer for generating a distribution over a vocabulary, a context layer for generating a distribution over the context, and a switch for generating a weighting between the distributions over the vocabulary and the context, generating a composite distribution based on the weighting, and selecting a word of an answer using the composite distribution.