公开(公告)号：US20220067534A1

公开(公告)日：2022-03-03

申请号：US17006570

申请日：2020-08-28

Applicant: salesforce.com, inc.

Inventor： Junwen Bai ,  Weiran Wang ,  Yingbo Zhou ,  Caiming Xiong

IPC: G06N3/08 ,  G06K9/62 ,  G06N3/04

Abstract: Embodiments described herein combine both masked reconstruction and predictive coding. Specifically, unlike contrastive learning, the mutual information between past states and future states are directly estimated. The context information can also be directly captured via shifted masked reconstruction—unlike standard masked reconstruction, the target reconstructed observations are shifted slightly towards the future to incorporate more predictability. The estimated mutual information and shifted masked reconstruction loss can then be combined as the loss function to update the neural model.

公开(公告)号：US12198060B2

公开(公告)日：2025-01-14

申请号：US17006570

申请日：2020-08-28

Applicant: Salesforce.com, Inc.

Inventor： Junwen Bai ,  Weiran Wang ,  Yingbo Zhou ,  Caiming Xiong

IPC: G06N3/088 ,  G06F18/21 ,  G06F18/214 ,  G06N3/049

Abstract: Embodiments described herein combine both masked reconstruction and predictive coding. Specifically, unlike contrastive learning, the mutual information between past states and future states are directly estimated. The context information can also be directly captured via shifted masked reconstruction—unlike standard masked reconstruction, the target reconstructed observations are shifted slightly towards the future to incorporate more predictability. The estimated mutual information and shifted masked reconstruction loss can then be combined as the loss function to update the neural model.