公开(公告)号：US20210173872A1

公开(公告)日：2021-06-10

申请号：US16869903

申请日：2020-05-08

Applicant: salesforce.com, inc.

Inventor： Samson Min Rong Tan ,  Shafiq Rayhan Joty

IPC: G06F16/9032 ,  G10L15/16 ,  G10L15/18 ,  G06F40/284

Abstract: Embodiments described herein provide systems and methods for generating an adversarial sample with inflectional perturbations for training a natural language processing (NLP) system. A natural language sentence is received at an inflection perturbation module. Tokens are generated from the natural language sentence. For each token that has a part of speech that is a verb, adjective, or an adverb, an inflected form is determined. An adversarial sample of the natural language sentence is generated by detokenizing inflected forms of the tokens. The NLP system is trained using the adversarial sample.

公开(公告)号：US20230237275A1

公开(公告)日：2023-07-27

申请号：US17830889

申请日：2022-06-02

Applicant: salesforce.com, inc.

Inventor： Guangsen Wang ,  Samson Min Rong Tan ,  Shafiq Rayhan Joty ,  Gang Wu ,  Chu Hong Hoi ,  Ka Chun Au

IPC: G06F40/35 ,  G06F40/40 ,  H04L51/02 ,  G06F40/186

CPC classification number: G06F40/35 ,  G06F40/40 ,  H04L51/02 ,  G06F40/186

Abstract: Embodiments provide a software framework for evaluating and troubleshooting real-world task-oriented bot systems. Specifically, the evaluation framework includes a generator that infers dialog acts and entities from bot definitions and generates test cases for the system via model-based paraphrasing. The framework may also include a simulator for task-oriented dialog user simulation that supports both regression testing and end-to-end evaluation. The framework may also include a remediator to analyze and visualize the simulation results, remedy some of the identified issues, and provide actionable suggestions for improving the task-oriented dialog system.

公开(公告)号：US20220164547A1

公开(公告)日：2022-05-26

申请号：US17150988

申请日：2021-01-15

Applicant: salesforce.com, inc.

Inventor： Samson Min Rong Tan ,  Shafiq Rayhan Joty

IPC: G06F40/45 ,  G06N3/08 ,  G06F40/289 ,  G06F40/30

Abstract: Embodiments described herein provide adversarial attacks targeting the cross-lingual generalization ability of massive multilingual representations, demonstrating their effectiveness on multilingual models for natural language inference and question answering. An efficient adversarial training scheme can thus be implemented with the adversarial attacks, which takes the same number of steps as standard supervised training and show that it encourages language-invariance in representations, thereby improving both clean and robust accuracy.

公开(公告)号：US11755847B2

公开(公告)日：2023-09-12

申请号：US17150988

申请日：2021-01-15

Applicant: salesforce.com, inc.

Inventor： Samson Min Rong Tan ,  Shafiq Rayhan Joty

IPC: G06F40/45 ,  G06F40/30 ,  G06F40/289 ,  G06N3/08

CPC classification number: G06F40/45 ,  G06F40/289 ,  G06F40/30 ,  G06N3/08

Abstract: Embodiments described herein provide adversarial attacks targeting the cross-lingual generalization ability of massive multilingual representations, demonstrating their effectiveness on multilingual models for natural language inference and question answering. An efficient adversarial training scheme can thus be implemented with the adversarial attacks, which takes the same number of steps as standard supervised training and show that it encourages language-invariance in representations, thereby improving both clean and robust accuracy.

公开(公告)号：US11256754B2

公开(公告)日：2022-02-22

申请号：US16869903

申请日：2020-05-08

Applicant: salesforce.com, inc.

Inventor： Samson Min Rong Tan ,  Shafiq Rayhan Joty

IPC: G06F16/9032 ,  G06F40/284 ,  G10L15/18 ,  G10L15/16

Abstract: Embodiments described herein provide systems and methods for generating an adversarial sample with inflectional perturbations for training a natural language processing (NLP) system. A natural language sentence is received at an inflection perturbation module. Tokens are generated from the natural language sentence. For each token that has a part of speech that is a verb, adjective, or an adverb, an inflected form is determined. An adversarial sample of the natural language sentence is generated by detokenizing inflected forms of the tokens. The NLP system is trained using the adversarial sample.