公开(公告)号：US11630858B2

公开(公告)日：2023-04-18

申请号：US17408248

申请日：2021-08-20

Applicant: Roku, Inc.

Inventor： Mihailo M. Stojancic ,  Prashant Ramanathan ,  Peter Wendt ,  Jose Pio Pereira ,  Shashank Merchant

IPC: G06F16/00 ,  G06F16/48 ,  G06F16/41 ,  G06F16/28 ,  G06F16/783 ,  G06F16/951 ,  G06F16/22 ,  G06F16/901 ,  G06V10/46 ,  G06V20/40

Abstract: The overall architecture and details of a scalable video fingerprinting and identification system that is robust with respect to many classes of video distortions is described. In this system, a fingerprint for a piece of multimedia content is composed of a number of compact signatures, along with traversal hash signatures and associated metadata. Numerical descriptors are generated for features found in a multimedia clip, signatures are generated from these descriptors, and a reference signature database is constructed from these signatures. Query signatures are also generated for a query multimedia clip. These query signatures are searched against the reference database using a fast similarity search procedure, to produce a candidate list of matching signatures. This candidate list is further analyzed to find the most likely reference matches. Signature correlation is performed between the likely reference matches and the query clip to improve detection accuracy.

公开(公告)号：US11617001B2

公开(公告)日：2023-03-28

申请号：US17399507

申请日：2021-08-11

Applicant: Roku, Inc.

Inventor： Andrew Laurence Ashbacher ,  Steven Michael Cormie ,  Markus Kurt Peter Cremer ,  Shashank Merchant ,  Andreas Schmidt

IPC: H04N21/431 ,  H04N21/435 ,  H04N21/488 ,  H04N21/6547 ,  H04N21/234 ,  H04N21/235 ,  H04N21/258 ,  H04N21/266

Abstract: In one aspect, a method includes identifying an upcoming content-modification opportunity on a channel, where the upcoming content-modification opportunity relates to a modifiable content-segment transmitted by a content-distribution system on the channel, and where the modifiable content-segment includes an overlay content portion and a non-overlay content portion. The method also includes responsive to identifying the upcoming content-modification opportunity on the channel, identifying a content-presentation device tuned to the channel. The method also includes determining location data associated with the overlay content portion. The method also includes transmitting, to the identified content-presentation device, the determined location data to facilitate the content-presentation device performing a content-modification operation in connection with the identified upcoming content-modification opportunity on the channel, where performing the content-modification operation involves replacing only the non-overlay content portion with supplemental content.

公开(公告)号：US11564001B2

公开(公告)日：2023-01-24

申请号：US16441924

申请日：2019-06-14

Applicant: Roku, Inc.

Inventor： Mihailo M. Stojancic ,  Sunil Suresh Kulkarni ,  Shashank Merchant ,  Jose Pio Pereira ,  Oleksiy Bolgarov

IPC: H04N21/439 ,  H04N21/44 ,  H04N1/32 ,  H04N21/8352 ,  G10H1/00 ,  H04N21/414 ,  H04N21/422 ,  H04N21/4223 ,  H04N21/433 ,  H04N21/84 ,  H04N21/845

Abstract: A mobile device responds in real time to media content presented on a media device, such as a television. The mobile device captures temporal fragments of audio-video content on its microphone, camera, or both and generates corresponding audio-video query fingerprints. The query fingerprints are transmitted to a search server located remotely or used with a search function on the mobile device for content search and identification. Audio features are extracted and audio signal global onset detection is used for input audio frame alignment. Additional audio feature signatures are generated from local audio frame onsets, audio frame frequency domain entropy, and maximum change in the spectral coefficients. Video frames are analyzed to find a television screen in the frames, and a detected active television quadrilateral is used to generate video fingerprints to be combined with audio fingerprints for more reliable content identification.

公开(公告)号：US20220207083A1

公开(公告)日：2022-06-30

申请号：US17673281

申请日：2022-02-16

Applicant: Roku, Inc.

Inventor： Jose Pio Pereira ,  Mihailo M. Stojancic ,  Shashank Merchant

IPC: G06F16/783 ,  G06F16/71 ,  G06F16/901 ,  G06T7/11 ,  G06T7/12 ,  G06T7/162 ,  G06V20/40 ,  G06T7/10

Abstract: A multi-dimensional database and indexes and operations on the multi-dimensional database are described which include video search applications or other similar sequence or structure searches. Traversal indexes utilize highly discriminative information about images and video sequences or about object shapes. Global and local signatures around keypoints are used for compact and robust retrieval and discriminative information content of images or video sequences of interest. For other objects or structures relevant signature of pattern or structure are used for traversal indexes. Traversal indexes are stored in leaf nodes along with distance measures and occurrence of similar images in the database. During a sequence query, correlation scores are calculated for single frame, for frame sequence, and video clips, or for other objects or structures.

公开(公告)号：US20220191588A1

公开(公告)日：2022-06-16

申请号：US17644660

申请日：2021-12-16

Applicant: Roku, Inc.

Inventor： Andreas Schmidt ,  Shashank Merchant ,  Markus Kurt Peter Cremer

IPC: H04N21/442 ,  H04N21/654

Abstract: Disclosed is a method that includes a media presentation device (i) generating query fingerprint data representing video frames of a channel being rendered by the media presentation device, and transmitting the generated query fingerprint data from the device over a network to a server, (ii) responsive to transmitting the query fingerprint data to the server, receiving, from the server, channel-icon fingerprint data representing channel icons of multiple channels selected based at least on the server having detected that the query fingerprint data representing the video frames of the channel being rendered by the media presentation device matches the reference fingerprint data representing the frames of the multiple channels of video content, and (iii) responsive to receiving the channel-icon fingerprint data from the server, performing channel-icon fingerprint matching to determine which channel in the multiple channels is the channel being rendered by the media presentation device.

公开(公告)号：US20220109912A1

公开(公告)日：2022-04-07

申请号：US17644120

申请日：2021-12-14

Applicant: Roku, Inc.

Inventor： Shashank Merchant ,  Do Phuoc Hoang Tuong Lan

IPC: H04N21/4402 ,  H04N21/4627 ,  H04N21/485 ,  G06F16/783

Abstract: The disclosure provides for use of steganographically-encoded data as a basis to control content modification. A server could determine through a fingerprint comparison that a portion of media content on a channel being rendered by a content presentation device matches at least one modifiable-content segment, such as at least one replaceable ad. The server could then use watermark data encoded in the media content of the channel as a basis to determine an actual identity of a modifiable-content segment present on the channel, so as to then control content modification.

公开(公告)号：US11288314B2

公开(公告)日：2022-03-29

申请号：US16442414

申请日：2019-06-14

Applicant: Roku, Inc.

Inventor： Jose Pio Pereira ,  Mihailo M. Stojancic ,  Shashank Merchant

IPC: G06F16/00 ,  G06F16/783 ,  G06F16/71 ,  G06F16/901 ,  G06K9/00 ,  G06T7/11 ,  G06T7/12 ,  G06T7/162 ,  G06T7/10

Abstract: A multi-dimensional database and indexes and operations on the multi-dimensional database are described which include video search applications or other similar sequence or structure searches. Traversal indexes utilize highly discriminative information about images and video sequences or about object shapes. Global and local signatures around keypoints are used for compact and robust retrieval and discriminative information content of images or video sequences of interest. For other objects or structures relevant signature of pattern or structure are used for traversal indexes. Traversal indexes are stored in leaf nodes along with distance measures and occurrence of similar images in the database. During a sequence query, correlation scores are calculated for single frame, for frame sequence, and video clips, or for other objects or structures.

公开(公告)号：US11288313B2

公开(公告)日：2022-03-29

申请号：US16442398

申请日：2019-06-14

Applicant: Roku, Inc.

Inventor： Jose Pio Pereira ,  Mihailo M. Stojancic ,  Shashank Merchant

IPC: G06F16/00 ,  G06F16/783 ,  G06F16/71 ,  G06F16/901 ,  G06K9/00 ,  G06T7/11 ,  G06T7/12 ,  G06T7/162 ,  G06T7/10

Abstract: A multi-dimensional database and indexes and operations on the multi-dimensional database are described which include video search applications or other similar sequence or structure searches. Traversal indexes utilize highly discriminative information about images and video sequences or about object shapes. Global and local signatures around keypoints are used for compact and robust retrieval and discriminative information content of images or video sequences of interest. For other objects or structures relevant signature of pattern or structure are used for traversal indexes. Traversal indexes are stored in leaf nodes along with distance measures and occurrence of similar images in the database. During a sequence query, correlation scores are calculated for single frame, for frame sequence, and video clips, or for other objects or structures.

公开(公告)号：US11281718B2

公开(公告)日：2022-03-22

申请号：US16442404

申请日：2019-06-14

Applicant: Roku, Inc.

Inventor： Jose Pio Pereira ,  Mihailo M. Stojancic ,  Shashank Merchant

IPC: G06F16/00 ,  G06F16/783 ,  G06F16/71 ,  G06F16/901 ,  G06K9/00 ,  G06T7/11 ,  G06T7/12 ,  G06T7/162 ,  G06T7/10

Abstract: A multi-dimensional database and indexes and operations on the multi-dimensional database are described which include video search applications or other similar sequence or structure searches. Traversal indexes utilize highly discriminative information about images and video sequences or about object shapes. Global and local signatures around keypoints are used for compact and robust retrieval and discriminative information content of images or video sequences of interest. For other objects or structures relevant signature of pattern or structure are used for traversal indexes. Traversal indexes are stored in leaf nodes along with distance measures and occurrence of similar images in the database. During a sequence query, correlation scores are calculated for single frame, for frame sequence, and video clips, or for other objects or structures.

公开(公告)号：US20210409807A1

公开(公告)日：2021-12-30

申请号：US17180263

申请日：2021-02-19

Applicant: Roku, Inc.

Inventor： Shashank Merchant ,  Bruce William Randall ,  Brian Johnson ,  Chungwon Seo ,  Seunghyeong Lee ,  Paul Jeffrey Holmquist

IPC: H04N21/43 ,  G06K9/00 ,  G06K9/62 ,  H04N21/8547 ,  H04N21/438 ,  H04N21/4402

Abstract: One high-level aspect of a content-modification system and related methods may involve facilitating content modification by a content-presentation device of received broadcast content in a controlled manner, while under circumstances in which some events that can impact timing may be unpredictable. In particular, certain operations by a content-presentation device may involve matching received content with specific expected content as determined by one or another component of the content-modification system, in order to confirm proper conditions are met for the content-presentation device to proceed with, or continue, content-modification operations. It can happen the matching procedure becomes subject or susceptible to timing irregularities, or jitter. In some instances, jitter may impact the ability to derive the benefits of content modification. Accordingly, example embodiments herein are directed to systems and method for compensation and/or mitigating the effects of jitter.