公开(公告)号：US20220265184A1

公开(公告)日：2022-08-25

申请号：US17472191

申请日：2021-09-10

Applicant: INSTITUTE OF AUTOMATION, CHINESE ACADEMY OF SCIENCES

Inventor： Jianhua TAO ,  Cong CAI ,  Bin LIU ,  Mingyue NIU

IPC: A61B5/16 ,  G06K9/00 ,  G06K9/62 ,  G06T7/00 ,  G10L25/30 ,  G10L25/57 ,  G10L25/63 ,  G10L25/66 ,  G06N3/08 ,  A61B5/00

Abstract: Disclosed is an automatic depression detection method using audio-video, including: acquiring original data containing two modalities of long-term audio file and long-term video file from an audio-video file; dividing the long-term audio file into several audio segments, and meanwhile dividing the long-term video file into a plurality of video segments; inputting each audio segment/each video segment into an audio feature extraction network/a video feature extraction network to obtain in-depth audio features/in-depth video features; calculating the in-depth audio features and the in-depth video features by using multi-head attention mechanism so as to obtain attention audio features and attention video features; aggregating the attention audio features and the attention video features into audio-video features; and inputting the audio-video features into a decision network to predict a depression level of an individual in the audio-video file.