公开(公告)号：US11580975B2

公开(公告)日：2023-02-14

申请号：US17014458

申请日：2020-09-08

Applicant: salesforce.com, inc.

Inventor： Weishi Wang ,  Shafiq Rayhan Joty ,  Chu Hong Hoi

IPC: G10L15/22 ,  G10L15/16 ,  G10L15/18 ,  G10L15/06 ,  G10L15/02 ,  G06K9/62

Abstract: Embodiments described herein provide a dynamic topic tracking mechanism that tracks how the conversation topics change from one utterance to another and use the tracking information to rank candidate responses. A pre-trained language model may be used for response selection in the multi-party conversations, which consists of two steps: (1) a topic-based pre-training to embed topic information into the language model with self-supervised learning, and (2) a multi-task learning on the pretrained model by jointly training response selection and dynamic topic prediction and disentanglement tasks.

公开(公告)号：US11782686B2

公开(公告)日：2023-10-10

申请号：US17459968

申请日：2021-08-27

Applicant: salesforce.com, inc.

Inventor： Yue Wang ,  Weishi Wang ,  Shafiq Rayhan Joty ,  Chu Hong Hoi

IPC: G06F9/44 ,  G06F8/41 ,  G06F40/20 ,  G06N3/084 ,  G06F18/214 ,  G06N3/047

CPC classification number: G06F8/427 ,  G06F18/214 ,  G06F40/20 ,  G06N3/047 ,  G06N3/084

Abstract: Embodiments described herein a code generation and understanding model that builds on a Transformer-based encoder-decoder framework. The code generation and understanding model is configured to derive generic representations for programming language (PL) and natural language (NL) in code domain via pre-training on unlabeled code corpus, and then to benefit many code-related downstream tasks with fine-tuning. Apart from the denoising sequence-to-sequence objectives widely adopted for pre-training on natural language, identifier tagging and prediction pre-training objective is adopted to enable the model to better leverage the crucial token type information from PL, which specifically are the identifiers assigned by developers.

公开(公告)号：US20210375280A1

公开(公告)日：2021-12-02

申请号：US17014458

申请日：2020-09-08

Applicant: salesforce.com, inc.

Inventor： Weishi Wang ,  Shafiq Rayhan Joty ,  Chu Hong Hoi

IPC: G10L15/22 ,  G10L15/16 ,  G10L15/18 ,  G10L15/02 ,  G10L15/06 ,  G06K9/62

Abstract: Embodiments described herein provide a dynamic topic tracking mechanism that tracks how the conversation topics change from one utterance to another and use the tracking information to rank candidate responses. A pre-trained language model may be used for response selection in the multi-party conversations, which consists of two steps: (1) a topic-based pre-training to embed topic information into the language model with self-supervised learning, and (2) a multi-task learning on the pretrained model by jointly training response selection and dynamic topic prediction and disentanglement tasks.

公开(公告)号：US20220382527A1

公开(公告)日：2022-12-01

申请号：US17459968

申请日：2021-08-27

Applicant: salesforce.com, inc.

Inventor： Yue Wang ,  Weishi Wang ,  Shafiq Rayhan Joty ,  Chu Hong Hoi

IPC: G06F8/41 ,  G06K9/62 ,  G06N3/08 ,  G06N3/04 ,  G06F40/20

Abstract: Embodiments described herein a code generation and understanding model that builds on a Transformer-based encoder-decoder framework. The code generation and understanding model is configured to derive generic representations for programming language (PL) and natural language (NL) in code domain via pre-training on unlabeled code corpus, and then to benefit many code-related downstream tasks with fine-tuning. Apart from the denoising sequence-to-sequence objectives widely adopted for pre-training on natural language, identifier tagging and prediction pre-training objective is adopted to enable the model to better leverage the crucial token type information from PL, which specifically are the identifiers assigned by developers.