公开(公告)号：US20220108086A1

公开(公告)日：2022-04-07

申请号：US17159625

申请日：2021-01-27

Applicant: salesforce.com, inc.

Inventor： Chien-Sheng Wu ,  Wenhao Liu ,  Caiming Xiong ,  Linqing Liu

IPC: G06F40/56 ,  G06F40/205

Abstract: Dialogue summarization is challenging due to its multi-speaker standpoints, casual spoken language style, and limited labelled data. The embodiments are directed to a coarse-to-fine dialogue summarization model that improves abstractive dialogue summarization quality and enables granular controllability. A summary draft that includes key words for turns in a dialogue conversation history is created. The summary draft includes pseudo-labelled interrogative pronoun categories and noisy key phrases. The dialogue conversation history is divided into segments. A generate language model is trained to generate a segment summary for each dialogue segment using a portion of the summary draft that corresponds to at least one dialogue turn in the dialogue segment. A dialogue summary is generated using the generative language model trained using the summary draft.

公开(公告)号：US11620515B2

公开(公告)日：2023-04-04

申请号：US16716249

申请日：2019-12-16

Applicant: salesforce.com, inc.

Inventor： Linqing Liu ,  Caiming Xiong

IPC: G06N3/08 ,  G06F40/40 ,  G06F40/30 ,  G06N3/04

Abstract: Systems and methods are provided that employ knowledge distillation under a multi-task learning setting. In some embodiments, the systems and methods are implemented with a larger teacher model and a smaller student model, each of which comprise one or more shared layers and a plurality of task layers for performing multiple tasks. During training of the teacher model, its shared layers are initialized, and then the teacher model is multi-task refined. The teacher model predicts teacher logits. During training of the student model, its shared layers are initialized. Knowledge distillation is employed to transfer knowledge from the teacher model to the student model by the student model updating its shared layers and task layers, for example, according to the teacher logits of the teacher model. Other features are also provided.

公开(公告)号：US20220129626A1

公开(公告)日：2022-04-28

申请号：US17080478

申请日：2020-10-26

Applicant: salesforce.com, inc.

Inventor： Linqing Liu ,  Caiming Xiong

IPC: G06F40/20 ,  G06N3/04

Abstract: Embodiments described herein propose a densely connected Transformer architecture in which each Transformer layer takes advantages of all previous layers. Specifically, the input for each Transformer layer comes from the outputs of all its preceding layers; and the output information of each layer will be incorporated in all its subsequent layers. In this way, a L-layer Transformer network will have L(L+1)/2 connections. In this way, the dense connection allows the linguistic information learned by the lower layer to be directly propagated to all upper layers and encourages feature reuse throughout the network. Each layer is thus directly optimized from the loss function in the fashion of implicit deep supervision.