公开(公告)号：US12056928B2

公开(公告)日：2024-08-06

申请号：US17211055

申请日：2021-03-24

Applicant: YAHOO ASSETS LLC

Inventor： Topojoy Biswas ,  Avijit Shah ,  Deven Santosh Shah

IPC: G06V20/40 ,  G06F18/214 ,  G06V20/62 ,  G11B27/10 ,  H04N21/81 ,  H04N21/845 ,  G06V30/10

CPC classification number: G06V20/42 ,  G06F18/214 ,  G06V20/47 ,  G06V20/49 ,  G06V20/63 ,  G11B27/102 ,  H04N21/8146 ,  H04N21/845 ,  G06V20/44 ,  G06V30/10 ,  G06V2201/10

Abstract: The disclosed systems and methods provide a novel framework that provides mechanisms for performing cost-effective, accurate and scalable detection and recognition of fine-grained events. The framework functions by training high precision and high recall object/optical character recognition (OCR) models and aligning video frames to text commentaries of the videos (e.g., licensed play-by-play). The disclosed framework operates as a single algorithm that performs multimodal alignments between events/actions within videos and their prescribed text. Thus, the disclosed framework is able to scale to fine-grained action categories across different venues by delving into the key frames and key aspects of a video to identify particular actions performed by particular actors, thereby providing the novelty of fine-granted action detection and recognition.