公开(公告)号：US20200065651A1

公开(公告)日：2020-02-27

申请号：US16664508

申请日：2019-10-25

Applicant: salesforce.com, inc.

Inventor： Stephen Joseph Merity ,  Caiming Xiong ,  James Bradbury ,  Richard Socher

IPC: G06N3/04 ,  G06N7/00 ,  G06N3/08 ,  G06F17/27

Abstract: The technology disclosed provides a so-called “pointer sentinel mixture architecture” for neural network sequence models that has the ability to either reproduce a token from a recent context or produce a token from a predefined vocabulary. In one implementation, a pointer sentinel-LSTM architecture achieves state of the art language modeling performance of 70.9 perplexity on the Penn Treebank dataset, while using far fewer parameters than a standard softmax LSTM.

公开(公告)号：US10565493B2

公开(公告)日：2020-02-18

申请号：US15421016

申请日：2017-01-31

Applicant: salesforce.com, inc.

Inventor： Stephen Joseph Merity ,  Caiming Xiong ,  James Bradbury ,  Richard Socher

IPC: G06N3/04 ,  G06F17/27 ,  G06N3/08 ,  G06N7/00

Abstract: The technology disclosed provides a so-called “pointer sentinel mixture architecture” for neural network sequence models that has the ability to either reproduce a token from a recent context or produce a token from a predefined vocabulary. In one implementation, a pointer sentinel-LSTM architecture achieves state of the art language modeling performance of 70.9 perplexity on the Penn Treebank dataset, while using far fewer parameters than a standard softmax LSTM.

公开(公告)号：US20180336453A1

公开(公告)日：2018-11-22

申请号：US15953265

申请日：2018-04-13

Applicant: salesforce.com, inc.

Inventor： Stephen Joseph Merity ,  Richard Socher ,  James Bradbury ,  Caiming Xiong

IPC: G06N3/04 ,  G06N3/08

Abstract: A system automatically generates recurrent neural network (RNN) architectures for performing specific tasks, for example, machine translation. The system represents RNN architectures using a domain specific language (DSL). The system generates candidate RNN architectures. The system predicts performances of the generated candidate RNN architectures, for example, using a neural network. The system filters the candidate RNN architectures based on their predicted performance. The system generates code for selected a candidate architectures. The generated code represents an RNN that is configured to perform the specific task. The system executes the generated code, for example, to evaluate an RNN or to use the RNN in an application.

公开(公告)号：US12235790B2

公开(公告)日：2025-02-25

申请号：US17670368

申请日：2022-02-11

Applicant: salesforce.com, inc.

Inventor： Alexander Rosenberg Johansen ,  Bryan McCann ,  James Bradbury ,  Richard Socher

IPC: G06N20/00 ,  G06F15/76 ,  G06F18/21 ,  G06F18/24 ,  G06F18/241 ,  G06F18/2413 ,  G06F40/169 ,  G06F40/30 ,  G06N3/04 ,  G06N3/044 ,  G06N3/045 ,  G06N3/047 ,  G06N3/048 ,  G06N3/08 ,  G06N3/084 ,  G06N5/04 ,  G06V10/764 ,  G06V10/776 ,  G06V10/82

Abstract: The technology disclosed proposes using a combination of computationally cheap, less-accurate bag of words (BoW) model and computationally expensive, more-accurate long short-term memory (LSTM) model to perform natural processing tasks such as sentiment analysis. The use of cheap, less-accurate BoW model is referred to herein as “skimming”. The use of expensive, more-accurate LSTM model is referred to herein as “reading”. The technology disclosed presents a probability-based guider (PBG). PBG combines the use of BoW model and the LSTM model. PBG uses a probability thresholding strategy to determine, based on the results of the BoW model, whether to invoke the LSTM model for reliably classifying a sentence as positive or negative. The technology disclosed also presents a deep neural network-based decision network (DDN) that is trained to learn the relationship between the BoW model and the LSTM model and to invoke only one of the two models.

公开(公告)号：US12198047B2

公开(公告)日：2025-01-14

申请号：US17122894

申请日：2020-12-15

Applicant: Salesforce.com, inc.

Inventor： James Bradbury ,  Stephen Joseph Merity ,  Caiming Xiong ,  Richard Socher

IPC: G06N3/08 ,  G06F17/16 ,  G06F40/00 ,  G06F40/216 ,  G06F40/30 ,  G06F40/44 ,  G06N3/04 ,  G06N3/044 ,  G06N3/045 ,  G06N3/10 ,  G10L15/16 ,  G10L15/18 ,  G10L25/30

Abstract: The technology disclosed provides a quasi-recurrent neural network (QRNN) encoder-decoder model that alternates convolutional layers, which apply in parallel across timesteps, and minimalist recurrent pooling layers that apply in parallel across feature dimensions.

公开(公告)号：US11520998B2

公开(公告)日：2022-12-06

申请号：US16709330

申请日：2019-12-10

Applicant: salesforce.com, inc.

Inventor： James Bradbury

IPC: G06F40/58 ,  G06F40/284 ,  G06N3/08 ,  G06N3/04 ,  G06N5/00 ,  G06F40/44 ,  G06F40/211 ,  G06F40/216

Abstract: An attentional neural machine translation model is provided for the task of machine translation that, according to some embodiments, leverages the hierarchical structure of language to perform natural language processing without a priori annotation. Other features are also provided.

公开(公告)号：US11250311B2

公开(公告)日：2022-02-15

申请号：US15853570

申请日：2017-12-22

Applicant: salesforce.com, inc.

Inventor： Alexander Rosenberg Johansen ,  Bryan McCann ,  James Bradbury ,  Richard Socher

IPC: G06N3/04 ,  G06K9/62 ,  G06N20/00 ,  G06F15/76 ,  G06F40/30 ,  G06F40/16 ,  G06N3/08 ,  G06N5/04 ,  G06F40/169

Abstract: The technology disclosed proposes using a combination of computationally cheap, less-accurate bag of words (BoW) model and computationally expensive, more-accurate long short-term memory (LSTM) model to perform natural processing tasks such as sentiment analysis. The use of cheap, less-accurate BoW model is referred to herein as “skimming”. The use of expensive, more-accurate LSTM model is referred to herein as “reading”. The technology disclosed presents a probability-based guider (PBG). PBG combines the use of BoW model and the LSTM model. PBG uses a probability thresholding strategy to determine, based on the results of the BoW model, whether to invoke the LSTM model for reliably classifying a sentence as positive or negative. The technology disclosed also presents a deep neural network-based decision network (DDN) that is trained to learn the relationship between the BoW model and the LSTM model and to invoke only one of the two models.

公开(公告)号：US10565318B2

公开(公告)日：2020-02-18

申请号：US15901722

申请日：2018-02-21

Applicant: salesforce.com, inc.

Inventor： James Bradbury

IPC: G06F17/28 ,  G06F17/27 ,  G06N3/08 ,  G06N3/04 ,  G06N5/00

Abstract: We introduce an attentional neural machine translation model for the task of machine translation that accomplishes the longstanding goal of natural language processing to take advantage of the hierarchical structure of language without a priori annotation. The model comprises a recurrent neural network grammar (RNNG) encoder with a novel attentional RNNG decoder and applies policy gradient reinforcement learning to induce unsupervised tree structures on both the source sequence and target sequence. When trained on character-level datasets with no explicit segmentation or parse annotation, the model learns a plausible segmentation and shallow parse, obtaining performance close to an attentional baseline.