公开(公告)号：US20230237275A1

公开(公告)日：2023-07-27

申请号：US17830889

申请日：2022-06-02

Applicant: salesforce.com, inc.

Inventor： Guangsen Wang ,  Samson Min Rong Tan ,  Shafiq Rayhan Joty ,  Gang Wu ,  Chu Hong Hoi ,  Ka Chun Au

IPC: G06F40/35 ,  G06F40/40 ,  H04L51/02 ,  G06F40/186

CPC classification number: G06F40/35 ,  G06F40/40 ,  H04L51/02 ,  G06F40/186

Abstract: Embodiments provide a software framework for evaluating and troubleshooting real-world task-oriented bot systems. Specifically, the evaluation framework includes a generator that infers dialog acts and entities from bot definitions and generates test cases for the system via model-based paraphrasing. The framework may also include a simulator for task-oriented dialog user simulation that supports both regression testing and end-to-end evaluation. The framework may also include a remediator to analyze and visualize the simulation results, remedy some of the identified issues, and provide actionable suggestions for improving the task-oriented dialog system.