公开(公告)号：US11036783B2

公开(公告)日：2021-06-15

申请号：US16365577

申请日：2019-03-26

Applicant: Roku, Inc.

Inventor： Prashant Ramanathan ,  Jose Pio Pereira ,  Shashank Merchant ,  Mihailo M. Stojancic

IPC: G06F16/00 ,  G06F16/48 ,  G06F16/41 ,  G06F16/28 ,  G06F16/783 ,  G06F16/951 ,  G06F16/22 ,  G06F16/901 ,  G06K9/00 ,  G06K9/46

Abstract: The overall architecture and details of a scalable video fingerprinting and identification system that is robust with respect to many classes of video distortions is described. In this system, a fingerprint for a piece of multimedia content is composed of a number of compact signatures, along with traversal hash signatures and associated metadata. Numerical descriptors are generated for features found in a multimedia clip, signatures are generated from these descriptors, and a reference signature database is constructed from these signatures. Query signatures are also generated for a query multimedia clip. These query signatures are searched against the reference database using a fast similarity search procedure, to produce a candidate list of matching signatures. This candidate list is further analyzed to find the most likely reference matches. Signature correlation is performed between the likely reference matches and the query clip to improve detection accuracy.

公开(公告)号：US11706481B2

公开(公告)日：2023-07-18

申请号：US16441987

申请日：2019-06-14

Applicant: Roku, Inc.

Inventor： Mihailo M. Stojancic ,  Daniel H. Eakins ,  Shashank Merchant ,  Prashant Ramanathan ,  Jose Pio Pereira

IPC: H04N21/439 ,  H04N21/8352 ,  G10H1/00 ,  H04N21/414 ,  H04N21/422 ,  H04N21/4223 ,  H04N21/433 ,  H04N21/84 ,  H04N21/845 ,  H04N1/32 ,  H04N21/43

CPC classification number: H04N21/4394 ,  G10H1/0008 ,  H04N1/32 ,  H04N21/41407 ,  H04N21/4223 ,  H04N21/42203 ,  H04N21/439 ,  H04N21/4332 ,  H04N21/43072 ,  H04N21/8352 ,  H04N21/84 ,  H04N21/8456 ,  G10H2210/041 ,  G10H2210/051 ,  G10H2240/141 ,  G10H2250/235

Abstract: A mobile device responds in real time to media content presented on a media device, such as a television. The mobile device captures temporal fragments of audio-video content on its microphone, camera, or both and generates corresponding audio-video query fingerprints. The query fingerprints are transmitted to a search server located remotely or used with a search function on the mobile device for content search and identification. Audio features are extracted and audio signal global onset detection is used for input audio frame alignment. Additional audio feature signatures are generated from local audio frame onsets, audio frame frequency domain entropy, and maximum change in the spectral coefficients. Video frames are analyzed to find a television screen in the frames, and a detected active television quadrilateral is used to generate video fingerprints to be combined with audio fingerprints for more reliable content identification.

公开(公告)号：US11366847B1

公开(公告)日：2022-06-21

申请号：US16502857

申请日：2019-07-03

Applicant: Roku, Inc.

Inventor： Prashant Ramanathan ,  Mihailo M. Stojancic

IPC: G06F16/00 ,  G06F16/48 ,  G06F16/783 ,  G06F16/41 ,  G06F16/28 ,  G06F16/951 ,  G06F16/22 ,  G06F16/901 ,  G06V10/46 ,  G06V20/40

Abstract: The overall architecture and details of a scalable video fingerprinting and identification system that is robust with respect to many classes of video distortions is described. In this system, a fingerprint for a piece of multimedia content is composed of a number of compact signatures, along with traversal hash signatures and associated metadata. Numerical descriptors are generated for features found in a multimedia clip, signatures are generated from these descriptors, and a reference signature database is constructed from these signatures. Query signatures are also generated for a query multimedia clip. These query signatures are searched against the reference database using a fast similarity search procedure, to produce a candidate list of matching signatures. This candidate list is further analyzed to find the most likely reference matches. Signature correlation is performed between the likely reference matches and the query clip to improve detection accuracy.

公开(公告)号：US11334615B2

公开(公告)日：2022-05-17

申请号：US16553658

申请日：2019-08-28

Applicant: Roku, Inc.

Inventor： Prashant Ramanathan ,  Jose Pio Pereira ,  Shashank Merchant ,  Mihailo M. Stojancic

IPC: G06F16/00 ,  G06F16/48 ,  G06F16/41 ,  G06F16/28 ,  G06F16/783 ,  G06F16/951 ,  G06F16/22 ,  G06F16/901 ,  G06V10/46 ,  G06V20/40

Abstract: The overall architecture and details of a scalable video fingerprinting and identification system that is robust with respect to many classes of video distortions is described. In this system, a fingerprint for a piece of multimedia content is composed of a number of compact signatures, along with traversal hash signatures and associated metadata. Numerical descriptors are generated for features found in a multimedia clip, signatures are generated from these descriptors, and a reference signature database is constructed from these signatures. Query signatures are also generated for a query multimedia clip. These query signatures are searched against the reference database using a fast similarity search procedure, to produce a candidate list of matching signatures. This candidate list is further analyzed to find the most likely reference matches. Signature correlation is performed between the likely reference matches and the query clip to improve detection accuracy.

公开(公告)号：USRE48791E1

公开(公告)日：2021-10-26

申请号：US16597648

申请日：2019-10-09

Applicant: Roku, Inc.

Inventor： Sunil Suresh Kulkarni ,  Jose Pio Pereira ,  Pradipkumar Dineshbhai Gajjar ,  Shashank Merchant ,  Prashant Ramanathan ,  Mihailo M. Stojancic

IPC: G06F16/783 ,  G06F16/43 ,  G06F16/245 ,  G06F16/22 ,  G06F16/951

Abstract: An architecture for a multimedia search system is described. To perform similarity matching of multimedia query frames against reference content, reference database comprising of a cluster index using cluster keys to perform similarity matching and a multimedia index to perform sequence matching is built. Methods to update and maintain the reference database that enables addition and removal of the multimedia contents, including portions of multimedia content, from the reference database in a running system are described. Hierarchical multi-level partitioning methods to organize the reference database are presented. Smart partitioning of the reference multimedia content according to the nature of the multimedia content, and according to the popularity among the social media, that supports scalable fast multimedia identification is also presented. A caching mechanism for multimedia search queries in a centralized or in a decentralized distributed system and a client based local multimedia search system enabling multimedia tracking are described.

公开(公告)号：US11126650B2

公开(公告)日：2021-09-21

申请号：US16385575

申请日：2019-04-16

Applicant: Roku, Inc.

Inventor： Mihailo M. Stojancic ,  Prashant Ramanathan ,  Peter Wendt ,  Jose Pio Pereira

IPC: G06F16/00 ,  G06F16/48 ,  G06F16/41 ,  G06F16/28 ,  G06F16/783 ,  G06F16/951 ,  G06F16/22 ,  G06F16/901 ,  G06K9/00 ,  G06K9/46

Abstract: The overall architecture and details of a scalable video fingerprinting and identification system that is robust with respect to many classes of video distortions is described. In this system, a fingerprint for a piece of multimedia content is composed of a number of compact signatures, along with traversal hash signatures and associated metadata. Numerical descriptors are generated for features found in a multimedia clip, signatures are generated from these descriptors, and a reference signature database is constructed from these signatures. Query signatures are also generated for a query multimedia clip. These query signatures are searched against the reference database using a fast similarity search procedure, to produce a candidate list of matching signatures. This candidate list is further analyzed to find the most likely reference matches. Signature correlation is performed between the likely reference matches and the query clip to improve detection accuracy.

公开(公告)号：US11042585B2

公开(公告)日：2021-06-22

申请号：US16367921

申请日：2019-03-28

Applicant: Roku, Inc.

Inventor： Prashant Ramanathan ,  Jose Pio Pereira ,  Shashank Merchant ,  Mihailo M. Stojancic

IPC: G06F16/00 ,  G06F16/48 ,  G06F16/41 ,  G06F16/28 ,  G06F16/783 ,  G06F16/951 ,  G06F16/22 ,  G06F16/901 ,  G06K9/00 ,  G06K9/46

Abstract: The overall architecture and details of a scalable video fingerprinting and identification system that is robust with respect to many classes of video distortions is described. In this system, a fingerprint for a piece of multimedia content is composed of a number of compact signatures, along with traversal hash signatures and associated metadata. Numerical descriptors are generated for features found in a multimedia clip, signatures are generated from these descriptors, and a reference signature database is constructed from these signatures. Query signatures are also generated for a query multimedia clip. These query signatures are searched against the reference database using a fast similarity search procedure, to produce a candidate list of matching signatures. This candidate list is further analyzed to find the most likely reference matches. Signature correlation is performed between the likely reference matches and the query clip to improve detection accuracy.