公开(公告)号：US20200372319A1

公开(公告)日：2020-11-26

申请号：US16559196

申请日：2019-09-03

Applicant: salesforce.com, inc.

Inventor： Lichao SUN ,  Kazuma HASHIMOTO ,  Jia LI ,  Richard SOCHER ,  Caiming XIONG

IPC: G06N3/00 ,  G06N3/04 ,  G06N3/08 ,  G06F17/27

Abstract: A method for evaluating robustness of one or more target neural network models using natural typos. The method includes receiving one or more natural typo generation rules associated with a first task associated with a first input document type, receiving a first target neural network model, and receiving a first document and corresponding its ground truth labels. The method further includes generating one or more natural typos for the first document based on the one or more natural typo generation rules, and providing, to the first target neural network model, a test document generated based on the first document and the one or more natural typos as an input document to generate a first output. A robustness evaluation result of the first target neural network model is generated based on a comparison between the output and the ground truth labels.

公开(公告)号：US20230419050A1

公开(公告)日：2023-12-28

申请号：US18463019

申请日：2023-09-07

Applicant: salesforce.com, inc.

Inventor： Akari ASAI ,  Kazuma HASHIMOTO ,  Richard SOCHER ,  Caiming XIONG

IPC: G06F40/40

CPC classification number: G06F40/40 ,  G06F40/30

Abstract: Embodiments described herein provide a pipelined natural language question answering system that improves a BERT-based system. Specifically, the natural language question answering system uses a pipeline of neural networks each trained to perform a particular task. The context selection network identifies premium context from context for the question. The question type network identifies the natural language question as a yes, no, or span question and a yes or no answer to the natural language question when the question is a yes or no question. The span extraction model determines an answer span to the natural language question when the question is a span question.

公开(公告)号：US20210216728A1

公开(公告)日：2021-07-15

申请号：US17214691

申请日：2021-03-26

Applicant: salesforce.com, inc.

Inventor： Kazuma HASHIMOTO ,  Raffaella BUSCHIAZZO ,  James BRADBURY ,  Teresa MARSHALL ,  Caiming XIONG ,  Richard SOCHER

IPC: G06F40/58

Abstract: Approaches for the translation of structured text include an embedding module for encoding and embedding source text in a first language, an encoder for encoding output of the embedding module, a decoder for iteratively decoding output of the encoder based on generated tokens in translated text from previous iterations, a beam module for constraining output of the decoder with respect to possible embedded tags to include in the translated text for a current iteration using a beam search, and a layer for selecting a token to be included in the translated text for the current iteration. The translated text is in a second language different from the first language. In some embodiments, the approach further includes scoring and pointer modules for selecting the token based on the output of the beam module or copied from the source text or reference text from a training pair best matching the source text.

公开(公告)号：US20200372341A1

公开(公告)日：2020-11-26

申请号：US16695494

申请日：2019-11-26

Applicant: salesforce.com, inc.

Inventor： Akari ASAI ,  Kazuma HASHIMOTO ,  Richard SOCHER ,  Caiming XIONG

IPC: G06N3/08 ,  G06N3/04 ,  G06F40/40

Abstract: Embodiments described herein provide a pipelined natural language question answering system that improves a BERT-based system. Specifically, the natural language question answering system uses a pipeline of neural networks each trained to perform a particular task. The context selection network identifies premium context from context for the question. The question type network identifies the natural language question as a yes, no, or span question and a yes or no answer to the natural language question when the question is a yes or no question. The span extraction model determines an answer span to the natural language question when the question is a span question.

公开(公告)号：US20180121799A1

公开(公告)日：2018-05-03

申请号：US15421431

申请日：2017-01-31

Applicant: salesforce.com, inc.

Inventor： Kazuma HASHIMOTO ,  Caiming XIONG ,  Richard SOCHER

IPC: G06N3/08 ,  G06N3/04 ,  G06N3/063

CPC classification number: G06N3/04 ,  G06F17/20 ,  G06F17/2705 ,  G06F17/2715 ,  G06F17/274 ,  G06F17/277 ,  G06F17/2785 ,  G06N3/0445 ,  G06N3/0454 ,  G06N3/0472 ,  G06N3/063 ,  G06N3/08 ,  G06N3/084 ,  G10L15/16 ,  G10L15/18 ,  G10L25/30

Abstract: The technology disclosed provides a so-called “joint many-task neural network model” to solve a variety of increasingly complex natural language processing (NLP) tasks using growing depth of layers in a single end-to-end model. The model is successively trained by considering linguistic hierarchies, directly connecting word representations to all model layers, explicitly using predictions in lower tasks, and applying a so-called “successive regularization” technique to prevent catastrophic forgetting. Three examples of lower level model layers are part-of-speech (POS) tagging layer, chunking layer, and dependency parsing layer. Two examples of higher level model layers are semantic relatedness layer and textual entailment layer. The model achieves the state-of-the-art results on chunking, dependency parsing, semantic relatedness and textual entailment.

公开(公告)号：US20200184020A1

公开(公告)日：2020-06-11

申请号：US16264392

申请日：2019-01-31

Applicant: salesforce.com, inc.

Inventor： Kazuma HASHIMOTO ,  Raffaella BUSCHIAZZO ,  James BRADBURY ,  Teresa MARSHALL ,  Caiming XIONG ,  Richard SOCHER

IPC: G06F17/28

Abstract: Approaches for the translation of structured text include an embedding module for encoding and embedding source text in a first language, an encoder for encoding output of the embedding module, a decoder for iteratively decoding output of the encoder based on generated tokens in translated text from previous iterations, a beam module for constraining output of the decoder with respect to possible embedded tags to include in the translated text for a current iteration using a beam search, and a layer for selecting a token to be included in the translated text for the current iteration. The translated text is in a second language different from the first language. In some embodiments, the approach further includes scoring and pointer modules for selecting the token based on the output of the beam module or copied from the source text or reference text from a training pair best matching the source text.

公开(公告)号：US20180121788A1

公开(公告)日：2018-05-03

申请号：US15421424

申请日：2017-01-31

Applicant: salesforce.com, inc.

Inventor： Kazuma HASHIMOTO ,  Caiming XIONG ,  Richard SOCHER

IPC: G06N3/04 ,  G06F17/27

CPC classification number: G06N3/04 ,  G06F17/20 ,  G06F17/2705 ,  G06F17/2715 ,  G06F17/274 ,  G06F17/277 ,  G06F17/2785 ,  G06N3/0445 ,  G06N3/0454 ,  G06N3/0472 ,  G06N3/063 ,  G06N3/08 ,  G06N3/084 ,  G10L15/16 ,  G10L15/18 ,  G10L25/30

Abstract: The technology disclosed provides a so-called “joint many-task neural network model” to solve a variety of increasingly complex natural language processing (NLP) tasks using growing depth of layers in a single end-to-end model. The model is successively trained by considering linguistic hierarchies, directly connecting word representations to all model layers, explicitly using predictions in lower tasks, and applying a so-called “successive regularization” technique to prevent catastrophic forgetting. Three examples of lower level model layers are part-of-speech (POS) tagging layer, chunking layer, and dependency parsing layer. Two examples of higher level model layers are semantic relatedness layer and textual entailment layer. The model achieves the state-of-the-art results on chunking, dependency parsing, semantic relatedness and textual entailment.

公开(公告)号：US20180121787A1

公开(公告)日：2018-05-03

申请号：US15421407

申请日：2017-01-31

Applicant: salesforce.com, inc.

Inventor： Kazuma HASHIMOTO ,  Caiming XIONG ,  Richard SOCHER

IPC: G06N3/04 ,  G06F17/27

CPC classification number: G06N3/04 ,  G06F17/20 ,  G06F17/2705 ,  G06F17/2715 ,  G06F17/274 ,  G06F17/277 ,  G06F17/2785 ,  G06N3/0445 ,  G06N3/0454 ,  G06N3/0472 ,  G06N3/063 ,  G06N3/08 ,  G06N3/084 ,  G10L15/16 ,  G10L15/18 ,  G10L25/30

Abstract: The technology disclosed provides a so-called “joint many-task neural network model” to solve a variety of increasingly complex natural language processing (NLP) tasks using growing depth of layers in a single end-to-end model. The model is successively trained by considering linguistic hierarchies, directly connecting word representations to all model layers, explicitly using predictions in lower tasks, and applying a so-called “successive regularization” technique to prevent catastrophic forgetting. Three examples of lower level model layers are part-of-speech (POS) tagging layer, chunking layer, and dependency parsing layer. Two examples of higher level model layers are semantic relatedness layer and textual entailment layer. The model achieves the state-of-the-art results on chunking, dependency parsing, semantic relatedness and textual entailment.