公开(公告)号：US11025907B2

公开(公告)日：2021-06-01

申请号：US16289149

申请日：2019-02-28

Applicant: GOOGLE LLC

Inventor： Shan Li ,  Claudionor Coelho ,  Aki Kuusela ,  Dake He

IPC: H04N19/107 ,  H04N19/119 ,  H04N19/176 ,  H04N19/96 ,  G06N3/04 ,  G06N3/08

Abstract: Convolutional neural networks (CNN) that determine a mode decision (e.g., block partitioning) for encoding a block include feature extraction layers and multiple classifiers. A non-overlapping convolution operation is performed at a feature extraction layer by setting a stride value equal to a kernel size. The block has a N×N size, and a smallest partition output for the block has a S×S size. Classification layers of each classifier receive feature maps having a feature dimension. An initial classification layer receives the feature maps as an output of a final feature extraction layer. Each classifier infers partition decisions for sub-blocks of size (αS)×(αS) of the block, wherein α is a power of 2 and α=2, . . . , N/S, by applying, at some successive classification layers, a 1×1 kernel to reduce respective feature dimensions; and outputting by a last layer of the classification layers an output corresponding to a N/(αS)×N/(αS)×1 output map.

公开(公告)号：US20230412821A1

公开(公告)日：2023-12-21

申请号：US18252158

申请日：2020-11-09

Applicant: Google LLC

Inventor： Daniele Moro ,  Claudionor Coelho ,  Sean R. Purser-Haskell ,  Hao Zhuang ,  Stan Vitvitskyy

IPC: H04N19/176 ,  H04N19/154 ,  G06T7/11 ,  G06V10/764 ,  G06V30/414

CPC classification number: H04N19/176 ,  H04N19/154 ,  G06V30/414 ,  G06V10/764 ,  G06T7/11

Abstract: This document describes systems and techniques for encoding a video frame using different compression ratios or compression algorithms for text blocks and non-text blocks. The described systems and techniques can determine, using a machine-learned model, which blocks of a frame include and do not include text. The described systems and techniques can then use a different compression ratio or compression algorithm for text blocks than the compression ratio or compression algorithm used for non-text blocks. For example, the systems and techniques can encode the text blocks using a first compression ratio that results in higher video quality than a second compression ratio used on at least some non-text blocks. In this way, the described systems and techniques can improve text legibility in a video file without significantly increasing the bandwidth requirements to transmit the video file to remote computing devices.

公开(公告)号：US20220201316A1

公开(公告)日：2022-06-23

申请号：US17601639

申请日：2019-03-21

Applicant: Google LLC

Inventor： Claudionor Coelho ,  Aki Kuusela ,  Joseph Young ,  Shan Li ,  Dake He

IPC: H04N19/147 ,  H04N19/176 ,  H04N19/96 ,  G06T9/00

Abstract: An apparatus for encoding an image block includes a processor that presents, to a machine-learning model, the image block, obtains the partition decision for encoding the image block from the model, and encodes the image block using the partition decision. The model is trained to output a partition decision for encoding the image block by using training data for a plurality of training blocks as input, the training data including for a training block, partition decisions for encoding the training block, and, for each partition decision, a rate-distortion value resulting from encoding the training block using the partition decision. The model is trained using a loss function combining a partition loss function based upon a relationship between the partition decisions and respective predicted partitions, and a rate-distortion cost loss function based upon a relationship between the rate-distortion values and respective predicted rate-distortion values.

公开(公告)号：US20200275101A1

公开(公告)日：2020-08-27

申请号：US16868729

申请日：2020-05-07

Applicant: GOOGLE LLC

Inventor： Claudionor Coelho ,  Dake He ,  Aki Kuusela ,  Shan Li

IPC: H04N19/124 ,  H04N19/164 ,  H04N19/176 ,  H04N19/96

Abstract: Encoding an image block using a quantization parameter includes presenting, to an encoder that includes a machine-learning model, the image block and a value derived from the quantization parameter, where the value is a result of a non-linear function using the quantization parameter as input, where the non-linear function relates to a second function used to calculate, using the quantization parameter, a Lagrange multiplier that is used in a rate-distortion calculation, and where the machine-learning model is trained to output mode decision parameters for encoding the image block; obtaining the mode decision parameters from the encoder; and encoding, in a compressed bitstream, the image block using the mode decision parameters.

公开(公告)号：US10674152B2

公开(公告)日：2020-06-02

申请号：US16134134

申请日：2018-09-18

Applicant: GOOGLE LLC

Inventor： Claudionor Coelho ,  Dake He ,  Aki Kuusela ,  Shan Li

IPC: H04N19/124 ,  H04N19/164 ,  H04N19/96 ,  H04N19/176

Abstract: A method for encoding an image block includes presenting, to a machine-learning model, the image block and a first value corresponding to a first quantization parameter; obtaining first mode decision parameters from the machine-learning model; and encoding the image block using the first mode decision parameters. The first value results from a non-linear function using the first quantization parameter as input. The machine-learning model is trained to output mode decision parameters by using training data. Each training datum includes a training block that is encoded by a second encoder, second mode decision parameters used by the second encoder for encoding the training block, and a second value corresponding to a second quantization parameter. The second encoder used the second quantization parameter for encoding the training block and the second value results from the non-linear function using the second quantization parameter as input.