公开(公告)号：US20190163977A1

公开(公告)日：2019-05-30

申请号：US16171018

申请日：2018-10-25

Applicant: Sony Interactive Entertainment Inc.

Inventor： Ruxin Chen ,  Naveen Kumar ,  Haoqi Li

IPC: G06K9/00 ,  G06K9/62 ,  G06N20/00

Abstract: Methods and systems for performing sequence level prediction of a video scene are described. Video information in a video scene is represented as a sequence of features depicted each frame. An environment state for each time step t corresponding to each frame is represented by the video information for time step t and predicted affective information from a previous time step t−1. An action A(t) as taken with an agent controlled by a machine learning algorithm for the frame at step t, wherein an output of the action A(t) represents affective label prediction for the frame at the time step t. A pool of predicted actions is transformed to a predicted affective history at a next time step t+1. The predictive affective history is included as part of the environment state for the next time step t+1. A reward R is generated on predicted actions up to the current time step t, by comparing them against corresponding annotated movie scene affective labels.