公开(公告)号：US20210312264A1

公开(公告)日：2021-10-07

申请号：US17354430

申请日：2021-06-22

Applicant: Beijing Baidu Netcom Science and Technology Co., Ltd.

Inventor： Fukui YANG ,  Shengzhao WEN ,  Junyu HAN

IPC: G06N3/04 ,  G06K9/62 ,  G06F17/16

Abstract: A method, and an apparatus for model distillation are provided. The method may include: obtaining a batch of teacher features corresponding to a teacher model and a batch of student features corresponding to a student model; determining a set of teacher similarities corresponding to the batch of teacher features and a set of student similarities corresponding to the batch of student features; determining weights of loss values of features of images based on difference values corresponding to the images; and weighting a loss value of a feature of each image in a batch of images, training the student model by using a weighting result. The present disclosure may use the difference values between the feature similarities of the student model and the feature similarities of the teacher model to determine the weights of the loss values.