公开(公告)号：US20240111965A1

公开(公告)日：2024-04-04

申请号：US18476598

申请日：2023-09-28

Applicant: AJOU UNIVERSITY INDUSTRY-ACADEMIC COOPERATION FOUNDATION

Inventor： Tae-Sun Chung ,  Zhen Zhang

IPC: G06F40/40

CPC classification number: G06F40/40

Abstract: The electronic device for improving the inference performance of a pre-trained language model according to an exemplary embodiment of the present invention includes a processor for sequentially passing input data through a plurality of transformer layers of the pre-trained language model and obtaining output data, wherein the processor calculates a probability distribution for prediction results received from each transformer layer in each of a plurality of middle layers connected to each rear end of the plurality of transformer layers, and measures a confidence level of the prediction results based on an entropy value calculated by using the probability distribution, and wherein when a confidence level less than a predefined value is measured in a predefined number of consecutive middle layers among the plurality of middle layers, the processor outputs a prediction result of the first middle layer as the output data by taking the last first middle layer among the consecutive middle layers as an exit.