公开(公告)号：US12100418B2

公开(公告)日：2024-09-24

申请号：US17472511

申请日：2021-09-10

Applicant: INSTITUTE OF AUTOMATION, CHINESE ACADEMY OF SCIENCES

Inventor： Jianhua Tao ,  Zheng Lian ,  Bin Liu ,  Xuefei Liu

IPC: G10L25/63 ,  G06F18/25 ,  G06F40/166 ,  G06F40/211 ,  G06F40/216 ,  G06F40/284 ,  G06F40/289 ,  G06F40/30 ,  G06N20/00 ,  G06N20/20 ,  G06V20/40 ,  G06V40/16 ,  G10L15/02 ,  G10L15/26 ,  G10L25/30

CPC classification number: G10L25/63 ,  G06F18/253 ,  G06F40/166 ,  G06F40/211 ,  G06F40/216 ,  G06F40/284 ,  G06F40/289 ,  G06F40/30 ,  G06N20/00 ,  G06N20/20 ,  G06V20/41 ,  G06V40/166 ,  G06V40/168 ,  G10L15/02 ,  G10L15/26 ,  G10L25/30

Abstract: Disclosed is a dialogue emotion correction method based on a graph neural network, including: extracting acoustic features, text features, and image features from a video file to fuse them into multi-modal features; obtaining an emotion prediction result of each sentence of a dialogue in the video file by using the multi-modal features; fusing the emotion prediction result of each sentence with interaction information between talkers in the video file to obtain interaction information fused emotion features; combining, on the basis of the interaction information fused emotion features, with context-dependence relationship in the dialogue to obtain time-series information fused emotion features; correcting, by using the time-series information fused emotion features, the emotion prediction result of each sentence that is obtained previously as to obtain a more accurate emotion recognition result.