公开(公告)号：US12073178B2

公开(公告)日：2024-08-27

申请号：US17586504

申请日：2022-01-27

Applicant: salesforce.com, inc.

Inventor： Zahra Fatemi ,  Caiming Xiong ,  Wenhao Liu ,  Chen Xing

IPC: G06F40/279 ,  G06F40/35 ,  G06F40/56 ,  G06N3/045 ,  G06N3/08

CPC classification number: G06F40/279 ,  G06F40/35 ,  G06F40/56 ,  G06N3/045 ,  G06N3/08

Abstract: Embodiments are directed to a training framework for reducing gender bias in a pre-trained language model. To reduce gender bias a gender neutral dataset is generated. Next, parameters of the pre-trained language model are frozen and do not change during a subsequent training phase. As all the pre-trained parameters are frozen, forgetting of information from the original training data is minimized. New parameters are added to the language model. The new parameters may be associated with gender related terms, such as profession names. In a subsequent training phase the new parameters of the language model are trained using a gender neutral dataset.

公开(公告)号：US20230154213A1

公开(公告)日：2023-05-18

申请号：US17587161

申请日：2022-01-28

Applicant: salesforce.com, inc.

Inventor： Mingfei Gao ,  Chen Xing

IPC: G06V20/62 ,  G06T9/00 ,  G06V10/22 ,  G06V10/774 ,  G06V10/77 ,  G06T1/60 ,  G06F40/126

CPC classification number: G06V20/635 ,  G06F40/126 ,  G06T1/60 ,  G06T9/00 ,  G06V10/225 ,  G06V10/7715 ,  G06V10/7747

Abstract: Embodiments described herein provide methods and systems for open vocabulary object detection of images. given a pre-trained vision-language model and an image-caption pair, an activation map may be computed in the image that corresponds to an object of interest mentioned in the caption. The activation map is then converted into a pseudo bounding-box label for the corresponding object category. The open vocabulary detector is then directly supervised by these pseudo box-labels, which enables training object detectors with no human-provided bounding-box annotations.

公开(公告)号：US12072955B2

公开(公告)日：2024-08-27

申请号：US17532851

申请日：2021-11-22

Applicant: salesforce.com, inc.

Inventor： Chen Xing ,  Wenhao Liu ,  Chu Hong Hoi ,  Nitish Shirish Keskar ,  Caiming Xiong

IPC: G06F18/214 ,  G06F18/21 ,  G06F40/00

CPC classification number: G06F18/2148 ,  G06F18/2163 ,  G06F40/00

Abstract: Embodiments are directed to pre-training a transformer model using more parameters for sophisticated patterns (PSP++). The transformer model is divided into a held-out model and a main model. A forward pass and a backward pass are performed on the held-out model, where the forward pass determines self-attention hidden states of the held-out model and the backward pass determines loss of the held-out model. A forward pass on the main model is performed to determine a self-attention hidden states of the main model. The self-attention hidden states of the main model are concatenated with the self-attention hidden states of the held-out model. A backward pass is performed on the main model to determine a loss of the main model. The parameters of the held-out model are updated to reflect the loss of the held-out model and parameters of the main model are updated to reflect the loss of the main model.

公开(公告)号：US20220391640A1

公开(公告)日：2022-12-08

申请号：US17532851

申请日：2021-11-22

Applicant: salesforce.com, inc.

Inventor： Chen Xing ,  Wenhao Liu ,  Chu Hong Hoi ,  Nitish Shirish Keskar ,  Caiming Xiong

IPC: G06K9/62 ,  G06F40/00

Abstract: Embodiments are directed to pre-training a transformer model using more parameters for sophisticated patterns (PSP++). The transformer model is divided into a held-out model and a main model. A forward pass and a backward pass are performed on the held-out model, where the forward pass determines self-attention hidden states of the held-out model and the backward pass determines loss of the held-out model. A forward pass on the main model is performed to determine a self-attention hidden states of the main model. The self-attention hidden states of the main model are concatenated with the self-attention hidden states of the held-out model. A backward pass is performed on the main model to determine a loss of the main model. The parameters of the held-out model are updated to reflect the loss of the held-out model and parameters of the main model are updated to reflect the loss of the main model.