公开(公告)号：US11080595B2

公开(公告)日：2021-08-03

申请号：US15420801

申请日：2017-01-31

Applicant: salesforce.com, inc.

Inventor： James Bradbury ,  Stephen Joseph Merity ,  Caiming Xiong ,  Richard Socher

IPC: G06N3/04 ,  G06N3/08 ,  G06F40/30 ,  G06F40/44 ,  G06F40/216 ,  G06F17/16 ,  G06N3/10 ,  G10L15/16 ,  G10L15/18 ,  G10L25/30 ,  G06F40/00

Abstract: The technology disclosed provides a quasi-recurrent neural network (QRNN) encoder-decoder model that alternates convolutional layers, which apply in parallel across timesteps, and minimalist recurrent pooling layers that apply in parallel across feature dimensions.

公开(公告)号：US11056099B2

公开(公告)日：2021-07-06

申请号：US16562257

申请日：2019-09-05

Applicant: salesforce.com, inc.

Inventor： Yingbo Zhou ,  Caiming Xiong

IPC: G10L15/06 ,  G06N3/08 ,  G10L15/14 ,  G10L15/16 ,  G06N3/04 ,  G06N7/00 ,  G10L25/51

Abstract: The disclosed technology teaches a deep end-to-end speech recognition model, including using multi-objective learning criteria to train a deep end-to-end speech recognition model on training data comprising speech samples temporally labeled with ground truth transcriptions. The multi-objective learning criteria updates model parameters of the model over one thousand to millions of backpropagation iterations by combining, at each iteration, a maximum likelihood objective function that modifies the model parameters to maximize a probability of outputting a correct transcription and a policy gradient function that modifies the model parameters to maximize a positive reward defined based on a non-differentiable performance metric which penalizes incorrect transcriptions in accordance with their conformity to corresponding ground truth transcriptions; and upon convergence after a final backpropagation iteration, persisting the modified model parameters learned by using the multi-objective learning criteria with the model to be applied to further end-to-end speech recognition.

公开(公告)号：US11029694B2

公开(公告)日：2021-06-08

申请号：US16176955

申请日：2018-10-31

Applicant: salesforce.com, inc.

Inventor： Chih-Yao Ma ,  Caiming Xiong

IPC: G05D1/02 ,  B25J9/00 ,  B25J9/16 ,  G06F3/16 ,  G06N3/08 ,  G06N20/00 ,  G06N3/02

Abstract: An agent for navigating a mobile automated system is disclosed herein. The navigation agent receives a navigation instruction and visual information for one or more observed images. The navigation agent is provided or equipped with self-awareness, which provides or supports the following abilities: identifying which direction to go or proceed by determining the part of the instruction that corresponds to the observed images (visual grounding), and identifying which part of the instruction has been completed or ongoing and which part is potentially needed for the next action selection (textual grounding). In some embodiments, the navigation agent applies regularization to ensures that the grounded instruction can correctly be used to estimate the progress made towards the navigation goal (progress monitoring).

公开(公告)号：US20200301925A1

公开(公告)日：2020-09-24

申请号：US16894495

申请日：2020-06-05

Applicant: salesforce.com, inc.

Inventor： Victor Zhong ,  Caiming Xiong ,  Richard Socher

IPC: G06F16/2452 ,  G06N3/08 ,  G06N7/00 ,  G06N3/04 ,  G06N3/00

Abstract: A computing system uses neural networks to translate natural language queries to database queries. The computing system uses a plurality of machine learning based models, each machine learning model for generating a portion of the database query. The machine learning models use an input representation generated based on terms of the input natural language query, a set of columns of the database schema, and the vocabulary of a database query language, for example, structured query language SQL. The plurality of machine learning based models may include an aggregation classifier model for determining an aggregation operator in the database query, a result column predictor model for determining the result columns of the database query, and a condition clause predictor model for determining the condition clause of the database query. The condition clause predictor is based on reinforcement learning.

公开(公告)号：US10776581B2

公开(公告)日：2020-09-15

申请号：US15974118

申请日：2018-05-08

Applicant: salesforce.com, inc.

Inventor： Bryan McCann ,  Nitish Shirish Keskar ,  Caiming Xiong ,  Richard Socher

IPC: G06F40/30 ,  G06N3/08 ,  G06N5/04 ,  G06N3/04 ,  G06F40/56 ,  G06F16/242 ,  G06F16/33 ,  G06F16/332

Abstract: Approaches for multitask learning as question answering include an input layer for encoding a context and a question, a self-attention based transformer including an encoder and a decoder, a first bi-directional long-term short-term memory (biLSTM) for further encoding an output of the encoder, a long-term short-term memory (LSTM) for generating a context-adjusted hidden state from the output of the decoder and a hidden state, an attention network for generating first attention weights based on an output of the first biLSTM and an output of the LSTM, a vocabulary layer for generating a distribution over a vocabulary, a context layer for generating a distribution over the context, and a switch for generating a weighting between the distributions over the vocabulary and the context, generating a composite distribution based on the weighting, and selecting a word of an answer using the composite distribution.

公开(公告)号：US10546217B2

公开(公告)日：2020-01-28

申请号：US16399163

申请日：2019-04-30

Applicant: salesforce.com, inc.

Inventor： Evan Albright ,  Caiming Xiong

IPC: G06K9/62 ,  G06N3/08 ,  G06N3/04 ,  G06K9/20 ,  G06T11/60 ,  G06K9/00

Abstract: A computer system generates augmented training datasets to train neural network models. The computer system receives an initial training dataset comprising images for training a neural network model, and generates an augmented training dataset by modifying images from the first training dataset. The computer system identifies a representation of a target object against a background from the initial training dataset and extracts a portion of the image displaying the target object. The computer system generates samples for including in the augmented training dataset based on the image. For example, new images may be obtained by performing transformations on the portion of the image displaying the target object and/or by overlaying the transformed portion of the image over a different background. The modified images are included in the augmented training dataset used for training the neural network model to recognize the target object.

公开(公告)号：US20190355270A1

公开(公告)日：2019-11-21

申请号：US16006691

申请日：2018-06-12

Applicant: salesforce.com, inc.

Inventor： Bryan McCann ,  Nitish Shirish Keskar ,  Caiming Xiong ,  Richard Socher

IPC: G09B7/02

Abstract: Approaches for natural language processing include a multi-layer encoder for encoding words from a context and words from a question in parallel, a multi-layer decoder for decoding the encoded context and the encoded question, a pointer generator for generating distributions over the words from the context, the words from the question, and words in a vocabulary based on an output from the decoder, and a switch. The switch generates a weighting of the distributions over the words from the context, the words from the question, and the words in the vocabulary, generates a composite distribution based on the weighting of the distribution over the first words from the context, the distribution over the second words from the question, and the distribution over the words in the vocabulary, and selects words for inclusion in an answer using the composite distribution.

公开(公告)号：US20190295530A1

公开(公告)日：2019-09-26

申请号：US16027111

申请日：2018-07-03

Applicant: salesforce.com, inc.

Inventor： Ehsan Hosseini-Asl ,  Caiming Xiong ,  Yingbo Zhou ,  Richard Socher

IPC: G10L15/07 ,  G06N3/08 ,  G10L15/16 ,  G10L15/22

Abstract: A system for domain adaptation includes a domain adaptation model configured to adapt a representation of a signal in a first domain to a second domain to generate an adapted presentation and a plurality of discriminators corresponding to a plurality of bands of values of a domain variable. Each of the plurality of discriminators is configured to discriminate between the adapted representation and representations of one or more other signals in the second domain.

公开(公告)号：US20190286073A1

公开(公告)日：2019-09-19

申请号：US16054935

申请日：2018-08-03

Applicant: salesforce.com, inc.

Inventor： Ehsan Hosseini-Asl ,  Caiming Xiong ,  Yingbo Zhou ,  Richard Socher

IPC: G05B13/02

Abstract: A method for training parameters of a first domain adaptation model includes evaluating a cycle consistency objective using a first task specific model associated with a first domain and a second task specific model associated with a second domain. The evaluating the cycle consistency objective is based on one or more first training representations adapted from the first domain to the second domain by a first domain adaptation model and from the second domain to the first domain by a second domain adaptation model, and one or more second training representations adapted from the second domain to the first domain by the second domain adaptation model and from the first domain to the second domain by the first domain adaptation model. The method further includes evaluating a learning objective based on the cycle consistency objective, and updating parameters of the first domain adaptation model based on learning objective.

公开(公告)号：US20190130218A1

公开(公告)日：2019-05-02

申请号：US15801297

申请日：2017-11-01

Applicant: salesforce.com, inc.

Inventor： Evan Albright ,  Caiming Xiong

IPC: G06K9/62 ,  G06N3/08 ,  G06N3/04 ,  G06K9/00 ,  G06K9/20 ,  G06T11/60

CPC classification number: G06K9/6257 ,  G06K9/00664 ,  G06K9/2054 ,  G06K9/6262 ,  G06K9/627 ,  G06N3/04 ,  G06N3/08 ,  G06T11/60

Abstract: A computer system generates augmented training datasets to train neural network models. The computer system receives an initial training dataset comprising images for training a neural network model, and generates an augmented training dataset by modifying images from the first training dataset. The computer system identifies a representation of a target object against a background from the initial training dataset and extracts a portion of the image displaying the target object. The computer system generates samples for including in the augmented training dataset based on the image. For example, new images may be obtained by performing transformations on the portion of the image displaying the target object and/or by overlaying the transformed portion of the image over a different background. The modified images are included in the augmented training dataset used for training the neural network model to recognize the target object.