公开(公告)号：US20230319327A1

公开(公告)日：2023-10-05

申请号：US18021636

申请日：2022-06-08

Applicant: Google LLC

Inventor： Yilin Wang ,  Balineedu Adsumilli ,  Junjie Ke ,  Hossein Talebi ,  Joong Yim ,  Neil Birkbeck ,  Peyman Milanfar ,  Feng Yang

IPC: H04N21/234 ,  H04N19/154 ,  H04N21/466

CPC classification number: H04N21/23418 ,  H04N19/154 ,  H04N21/4668

Abstract: Methods, systems, and media for determining perceptual quality indicators of video content items are provided. In some embodiments, the method comprises: receiving a video content item; extracting a plurality of frames from the video content item; determining, using a first subnetwork of a deep neural network, a content quality indicator for each frame of the plurality of frames of the video content item; determining, using a second subnetwork of the deep neural network, a video distortion indicator for each frame of the plurality of frames of the video content item; determining, using a third subnetwork of the deep neural network, a compression sensitivity indicator for each frame of the plurality of frames of the video content item; generating a quality level for each frame of the plurality of frames of the video content item that concatenates the content quality indicator, the video distortion indicator, and the compression sensitivity indicator for that frame of the video content item; generating an overall quality level for video content item by aggregating the quality level of each frame of the plurality of frames; and causing a video recommendation to be presented based on the overall quality level of the video content item.

公开(公告)号：US20230130410A1

公开(公告)日：2023-04-27

申请号：US17918170

申请日：2020-04-17

Applicant: Google LLC

Inventor： Xiyang Luo ,  Feng Yang ,  Hossein Talebi

IPC: G06T9/00 ,  G06T3/40

Abstract: Methods, systems, and computer programs encoded on a computer storage medium, that relate to generating quantization tables that are used during digital image compression of a digital image. Multiple training images are obtained. A model can be trained using the training images to generate a quantization table that can be used during encoding of an input image. For each training image, a quantization table can be obtained using the model. Using the quantization table, an encoded digital image is obtained for the training image. Using the encoded digital image and the training image, an image quality loss and a compression loss can be determined. An overall loss of the model can be determined by combining the image quality loss and the compression loss for the training image. The model can be updated based on the overall loss.

公开(公告)号：US20230061517A1

公开(公告)日：2023-03-02

申请号：US17789323

申请日：2020-02-03

Applicant: Google LLC

Inventor： Feng Yang ,  Hui Fang

IPC: G06V20/00 ,  G06V10/82 ,  G06V10/40 ,  G06V10/771 ,  G06V10/774

Abstract: This document describes techniques and apparatuses for verifying the authenticity of images. In aspects, methods include receiving, by a decoder system (220), an image (210) to be verified; performing feature recognition on the received image to determine determined features (238) of the received image; generating a first output (236) defining values representing the determined features; decoding the received image, by a message decoding neural network (252), to extract a signature (254) embedded in the received image, the embedded signature representing recovered features (258) of the received image; generating a second output (256) defining values representing the recovered features; providing the first output and the second output to a manipulation detection neural network (272); and generating, by the manipulation detection neural network, an estimation of an authenticity of the received image utilizing at least the first output and the second output.

公开(公告)号：US20220301141A1

公开(公告)日：2022-09-22

申请号：US17612372

申请日：2020-08-06

Applicant: GOOGLE LLC

Inventor： Catherine Shyu ,  Xiyang Luo ,  Feng Yang ,  Junjie Ke ,  Yicong Tian ,  Chao-Hung Chen ,  Xia Li ,  Luying Li ,  Wenjing Kang ,  Shun-Chuan Chen

IPC: G06T7/00 ,  G06V10/82

Abstract: Systems, devices, methods, and computer readable medium for evaluating visual quality of digital content are disclosed. Methods can include training machine learning models on images. A request is received to evaluate quality of an image included in a current version of a digital component generated by the computing device. The machine learning models are deployed on the image to generate a score for each quality characteristic of the image. A weight is assigned to each score to generate weighted scores. The weighted scores are combined to generate a combined score for the image. The combined score is compared to one or more thresholds to generate a quality of the image.

公开(公告)号：US12249002B2

公开(公告)日：2025-03-11

申请号：US17764445

申请日：2019-12-05

Applicant: Google LLC

Inventor： Innfarn Yoo ,  Feng Yang ,  Xiyang Luo

IPC: G06T1/00 ,  G06N3/045 ,  G06T3/4046 ,  G06T9/00

Abstract: A computer-implemented method that provides watermark-based image reconstruction to compensate for lossy encoding schemes. The method can generate a difference image describing the data loss associated with encoding an image using a lossy encoding scheme. The difference image can be encoded as a message and embedded in the encoded image using a watermark and later extracted from the encoded image. The difference image can be added to the encoded image to reconstruct the original image. As an example, an input image encoded using a lossy JPEG compression scheme can be embedded with the lost data and later reconstructed, using the embedded data, to a fidelity level that is identical or substantially similar to the original.

公开(公告)号：US12230024B2

公开(公告)日：2025-02-18

申请号：US17762289

申请日：2019-11-26

Applicant: Google LLC

Inventor： Yilin Wang ,  Hossein Talebi ,  Peyman Milanfar ,  Feng Yang ,  Balineedu Adsumilli

IPC: G06V10/98 ,  G06N3/045 ,  G06V10/82 ,  G06V20/40

Abstract: A trained model is retrained for video quality assessment and used to identify sets of adaptive compression parameters for transcoding user generated video content. Using transfer learning, the model, which is initially trained for image object detection, is retrained for technical content assessment and then again retrained for video quality assessment. The model is then deployed into a transcoding pipeline and used for transcoding an input video stream of user generated content. The transcoding pipeline may be structured in one of several ways. In one example, a secondary pathway for video content analysis using the model is introduced into the pipeline, which does not interfere with the ultimate output of the transcoding should there be a network or other issue. In another example, the model is introduced as a library within the existing pipeline, which would maintain a single pathway, but ultimately is not expected to introduce significant latency.

公开(公告)号：US11887270B2

公开(公告)日：2024-01-30

申请号：US17787699

申请日：2021-07-01

Applicant: Google LLC

Inventor： Junjie Ke ,  Feng Yang ,  Qifei Wang ,  Yilin Wang ,  Peyman Milanfar

IPC: G06K9/00 ,  G06T3/00 ,  G06T3/40 ,  G06T7/00

CPC classification number: G06T3/0012 ,  G06T3/40 ,  G06T7/0002 ,  G06T2207/20016 ,  G06T2207/20081 ,  G06T2207/30168

Abstract: The technology employs a patch-based multi-scale Transformer (300) that is usable with various imaging applications. This avoids constraints on image fixed input size and predicts the quality effectively on a native resolution image. A native resolution image (304) is transformed into a multi-scale representation (302), enabling the Transformer's self-attention mechanism to capture information on both fine-grained detailed patches and coarse-grained global patches. Spatial embedding (316) is employed to map patch positions to a fixed grid, in which patch locations at each scale are hashed to the same grid. A separate scale embedding (318) is employed to distinguish patches coming from different scales in the multiscale representation. Self-attention (508) is performed to create a final image representation. In some instances, prior to performing self-attention, the system may prepend a learnable classification token (322) to the set of input tokens.

公开(公告)号：US20240020788A1

公开(公告)日：2024-01-18

申请号：US18256783

申请日：2021-03-24

Applicant: Google LLC

Inventor： Xiyang Luo ,  Feng Yang ,  Ce Liu ,  Huiwen Chang ,  Peyman Milanfar ,  Yinxiao Li

IPC: G06T1/00

CPC classification number: G06T1/0085 ,  G06T2201/0083

Abstract: Systems and methods of the present disclosure are directed to a computing system. The computing system can obtain a message vector and video data comprising a plurality of video frames. The computing system can process the input video with a transformation portion of a machine-learned watermark encoding model to obtain a three-dimensional feature encoding of the input video. The computing system can process the three-dimensional feature encoding of the input video and the message vector with an embedding portion of the machine-learned watermark encoding model to obtain spatial-temporal watermark encoding data descriptive of the message vector. The computing system can generate encoded video data comprising a plurality of encoded video frames, wherein at least one of the plurality of encoded video frames includes the spatial-temporal watermark encoding data.

公开(公告)号：US20230325961A1

公开(公告)日：2023-10-12

申请号：US18008544

申请日：2021-06-21

Applicant: Google LLC

Inventor： Dake He ,  Tianhao Zhang ,  Elnaz Barshan Tashnizi ,  Xiyang Luo ,  Huiwen Chang ,  Feng Yang ,  Ryan Matthew Haggarty

IPC: G06T1/00 ,  G06T7/11 ,  G06T3/40

CPC classification number: G06T1/005 ,  G06T7/11 ,  G06T3/40 ,  G06T2201/0083 ,  G06T2207/20081 ,  G06T2201/0065

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining a visually imperceptible or a visually perceptible watermark and outputting a result based on the determination. A watermark decoder receives an input image. The watermark decoder applies a decoder machine learning model to decode a watermarks at different levels of zoom. The water mark decoder determines whether a watermark was decoded to obtain a decoded watermark. The watermark decoder outputs a result based on the determination whether the watermark was decoded through application of the decoder machine learning model to the input image that includes outputting a zoomed output decoded through application of the decoder machine learning model to the input image.

公开(公告)号：US20230214953A1

公开(公告)日：2023-07-06

申请号：US18008285

申请日：2020-06-05

Applicant: Google LLC

Inventor： Innfarn Yoo ,  Xiyang Luo ,  Feng Yang ,  Ondrej Stava

IPC: G06T1/00 ,  G09C5/00

CPC classification number: G06T1/0028 ,  G09C5/00 ,  G06T2201/0065

Abstract: Systems and methods are directed to a computing system. The computing system can include one or more processors, a message embedding model, a message extraction model, and a first set of instructions that cause the computing system to perform operations including obtaining the three-dimensional image data and the message vector. The operations can include inputting three-dimensional image data and a message vector into the message embedding model to obtain encoded three-dimensional image data. The operations can include using the message extraction model to extract an embedded message from the encoded three-dimensional image data to obtain a reconstructed message vector. The operations can include evaluating a loss function for a difference between the reconstructed message vector and the message vector and modifying values for parameters of at least the message embedding model based on the loss function.