公开(公告)号：US20230368003A1

公开(公告)日：2023-11-16

申请号：US17740497

申请日：2022-05-10

Applicant: ADOBE INC.

Inventor： Jiuxiang Gu ,  Zihan Wang ,  Jason Wen Yong Kuen ,  Handong Zhao ,  Vlad Ion Morariu ,  Ruiyi Zhang ,  Ani Nenkova Nenkova ,  Tong Sun

IPC: G06N3/04 ,  G06F40/284

CPC classification number: G06N3/0481 ,  G06F40/284

Abstract: The technology described herein is directed to an adaptive sparse attention pattern that is learned during fine-tuning and deployed in a machine-learning model. In aspects, a row or a column in an attention matrix with an importance score for a task that is above a threshold importance score is identified. The important row or the column is included in an adaptive attention pattern used with a machine-learning model having a self-attention operation. In response to an input, a task-specific inference is generated for the input using the machine-learning model with the adaptive attention pattern.