-
121.
公开(公告)号:US11222253B2
公开(公告)日:2022-01-11
申请号:US15421424
申请日:2017-01-31
Applicant: salesforce.com, inc.
Inventor: Kazuma Hashimoto , Caiming Xiong , Richard Socher
IPC: G06N3/04 , G06N3/08 , G06F40/30 , G06F40/205 , G06F40/216 , G06F40/253 , G06F40/284 , G06N3/063 , G10L15/18 , G10L25/30 , G10L15/16 , G06F40/00
Abstract: The technology disclosed provides a so-called “joint many-task neural network model” to solve a variety of increasingly complex natural language processing (NLP) tasks using growing depth of layers in a single end-to-end model. The model is successively trained by considering linguistic hierarchies, directly connecting word representations to all model layers, explicitly using predictions in lower tasks, and applying a so-called “successive regularization” technique to prevent catastrophic forgetting. Three examples of lower level model layers are part-of-speech (POS) tagging layer, chunking layer, and dependency parsing layer. Two examples of higher level model layers are semantic relatedness layer and textual entailment layer. The model achieves the state-of-the-art results on chunking, dependency parsing, semantic relatedness and textual entailment.
-
公开(公告)号:US20210374524A1
公开(公告)日:2021-12-02
申请号:US17098007
申请日:2020-11-13
Applicant: salesforce.com, inc.
Inventor: Yihao Feng , Caiming Xiong
Abstract: Some embodiments of the current disclosure disclose methods and systems for detecting out-of-distribution (ODD) data. For example, a method for detecting ODD data includes obtaining, at a neural network composed of a plurality of layers, a set of training data generated according to a distribution. Further, the method comprises generating, via a processor, a feature map by combining mapping functions corresponding to the plurality of layers into a vector of mapping function elements and mapping, by the feature map, the set of training data to a set of feature space training data in a feature space. Further, the method comprises identifying, via the processor, a hyper-ellipsoid in the feature space enclosing the feature space training data based on the generated feature map. In addition, the method comprises determining, via the processor, the first test data sample is OOD data when a mapped first test data sample in the feature space is outside the hyper-ellipsoid.
-
123.
公开(公告)号:US20210374353A1
公开(公告)日:2021-12-02
申请号:US17005316
申请日:2020-08-28
Applicant: salesforce.com, inc.
Inventor: Jianguo Zhang , Kazuma Hashimoto , Chien-Sheng Wu , Wenhao Liu , Richard Socher , Caiming Xiong
Abstract: An online system allows user interactions using natural language expressions. The online system uses a machine learning based model to infer an intent represented by a user expression. The machine learning based model takes as input a user expression and an example expression to compute a score indicating whether the user expression matches the example expression. Based on the scores, the intent inference module determines a most applicable intent for the expression. The online system determines a confidence threshold such that user expressions indicating a high confidence are assigned the most applicable intent and user expressions indicating a low confidence are assigned an out-of-scope intent. The online system encodes the example expressions using the machine learning based model. The online system may compare an encoded user expression with encoded example expressions to identify a subset of example expressions used to determine the most applicable intent.
-
公开(公告)号:US11170287B2
公开(公告)日:2021-11-09
申请号:US15881582
申请日:2018-01-26
Applicant: salesforce.com, inc.
Inventor: Victor Zhong , Caiming Xiong , Richard Socher
Abstract: A computer-implemented method for dual sequence inference using a neural network model includes generating a codependent representation based on a first input representation of a first sequence and a second input representation of a second sequence using an encoder of the neural network model and generating an inference based on the codependent representation using a decoder of the neural network model. The neural network model includes a plurality of model parameters learned according to a machine learning process. The encoder includes a plurality of coattention layers arranged sequentially, each coattention layer being configured to receive a pair of layer input representations and generate one or more summary representations, and an output layer configured to receive the one or more summary representations from a last layer among the plurality of coattention layers and generate the codependent representation.
-
公开(公告)号:US20210286369A1
公开(公告)日:2021-09-16
申请号:US17332756
申请日:2021-05-27
Applicant: salesforce.com, inc.
Inventor: Chih-Yao Ma , Caiming Xiong
Abstract: An agent for navigating a mobile automated system is disclosed herein. The navigation agent receives a navigation instruction and visual information for one or more observed images. The navigation agent is provided or equipped with self-awareness, which provides or supports the following abilities: identifying which direction to go or proceed by determining the part of the instruction that corresponds to the observed images (visual grounding), and identifying which part of the instruction has been completed or ongoing and which part is potentially needed for the next action selection (textual grounding). In some embodiments, the navigation agent applies regularization to ensures that the grounded instruction can correctly be used to estimate the progress made towards the navigation goal (progress monitoring).
-
公开(公告)号:US20210150365A1
公开(公告)日:2021-05-20
申请号:US16877325
申请日:2020-05-18
Applicant: salesforce.com, inc.
Inventor: Govardana Sachithanandam Ramachandran , Ka Chun Au , Shashank Harinath , Wenhao Liu , Alexis Roos , Caiming Xiong
Abstract: An embodiment provided herein preprocesses the input samples to the classification neural network, e.g., by adding Gaussian noise to word/sentence representations to make the function of the neural network satisfy Lipschitz property such that a small change in the input does not cause much change to the output if the input sample is in-distribution. Method to induce properties in the feature representation of neural network such that for out-of-distribution examples the feature representation magnitude is either close to zero or the feature representation is orthogonal to all class representations. Method to generate examples that are structurally similar to in-domain and semantically out-of domain for use in out-of-domain classification training. Method to prune feature representation dimension to mitigate long tail error of unused dimension in out-of-domain classification. Using these techniques, the accuracy of both in-domain and out-of-distribution identification can be improved.
-
127.
公开(公告)号:US20210141865A1
公开(公告)日:2021-05-13
申请号:US16680323
申请日:2019-11-11
Applicant: salesforce.com, inc.
Inventor: Michael Machado , James Douglas Harrison , Caiming Xiong , Xinyi Yang , Thomas Archie Cook , Roojuta Lalani , Jean-Marc Soumet , Karl Ryszard Skucha , Juan Manuel Rodriguez , Manju Vijayakumar , Vishal Motwani , Tian Xie , Bryan McCann , Nitish Shirish Keskar , Armen Abrahamyan , Zhihao Zou , Chitra Gulabrani , Minal Khodani , Adarsha Badarinath , Rohiniben Thakar , Srikanth Kollu , Kevin Schoen , Qiong Liu , Amit Hetawal , Kevin Zhang , Kevin Zhang , Victor Brouk , Johnson Liu , Rafael Amsili
Abstract: A multi-tenant system performs custom configuration of a tenant-specific chatbot to process and act upon natural language requests. The multi-tenant system configures the tenant-specific chatbots without requiring tenant-specific training. The multi-tenant system providing a user interface for configuring a tenant-specific set of permitted actions. The multi-tenant system determines a set of example phrases for each of the selected permitted actions. The multi-tenant system receives a natural language request from a user and identifies the action that the user wants to perform. The multi-tenant system uses a neural network to compare the natural language request with example phrases to identify an example phrase that matches the natural language request. The multi-tenant system performs the action corresponding to the matching example phrase.
-
公开(公告)号:US10963652B2
公开(公告)日:2021-03-30
申请号:US16264392
申请日:2019-01-31
Applicant: salesforce.com, inc.
Inventor: Kazuma Hashimoto , Raffaella Buschiazzo , James Bradbury , Teresa Marshall , Caiming Xiong , Richard Socher
Abstract: Approaches for the translation of structured text include an embedding module for encoding and embedding source text in a first language, an encoder for encoding output of the embedding module, a decoder for iteratively decoding output of the encoder based on generated tokens in translated text from previous iterations, a beam module for constraining output of the decoder with respect to possible embedded tags to include in the translated text for a current iteration using a beam search, and a layer for selecting a token to be included in the translated text for the current iteration. The translated text is in a second language different from the first language. In some embodiments, the approach further includes scoring and pointer modules for selecting the token based on the output of the beam module or copied from the source text or reference text from a training pair best matching the source text.
-
129.
公开(公告)号:US20210042604A1
公开(公告)日:2021-02-11
申请号:US17080656
申请日:2020-10-26
Applicant: salesforce.com, inc.
Inventor: Kazuma Hashimoto , Caiming Xiong , Richard SOCHER
IPC: G06N3/04 , G06N3/08 , G06F40/205 , G06F40/284 , G06F40/253 , G06F40/216 , G06N3/063 , G06F40/30 , G10L15/16 , G06F40/00 , G10L15/18 , G10L25/30
Abstract: The technology disclosed provides a so-called “joint many-task neural network model” to solve a variety of increasingly complex natural language processing (NLP) tasks using growing depth of layers in a single end-to-end model. The model is successively trained by considering linguistic hierarchies, directly connecting word representations to all model layers, explicitly using predictions in lower tasks, and applying a so-called “successive regularization” technique to prevent catastrophic forgetting. Three examples of lower level model layers are part-of-speech (POS) tagging layer, chunking layer, and dependency parsing layer. Two examples of higher level model layers are semantic relatedness layer and textual entailment layer. The model achieves the state-of-the-art results on chunking, dependency parsing, semantic relatedness and textual entailment.
-
公开(公告)号:US20200372339A1
公开(公告)日:2020-11-26
申请号:US16592474
申请日:2019-10-03
Applicant: salesforce.com, inc.
Inventor: Tong Che , Caiming Xiong
Abstract: Verification of discriminative models includes receiving an input; receiving a prediction from a discriminative model for the input; encoding, using an encoder, a latent variable based on the input; decoding, using a decoder, a reconstructed input based on the prediction and the latent variable; and determining, using an anomaly detection module, whether the prediction is reliable based on the input, the reconstructed input, and the latent variable. The encoder and the decoder are jointly trained to maximize an evidence lower bound of the encoder and the decoder. In some embodiments, the encoder and the decoder are further trained using a disentanglement constraint between the prediction and the latent variable. In some embodiments, the encoder and the decoder are further trained without using inputs that are out of a distribution of inputs used to train the discriminative model or that are adversarial to the discriminative model.
-
-
-
-
-
-
-
-
-