公开(公告)号：US20200092552A1

公开(公告)日：2020-03-19

申请号：US16134165

申请日：2018-09-18

Applicant: GOOGLE LLC

Inventor： Claudionor Coelho ,  Aki Kuusela ,  Shan Li ,  Dake He

IPC: H04N19/119 ,  H04N19/176 ,  H04N19/147 ,  H04N19/19

Abstract: A convolutional neural network (CNN) for determining a partitioning of a block is disclosed. The block is of size N×N and a smallest partition is of size S×S. The CNN includes feature extraction layers; a concatenation layer that receives, from the feature extraction layers, first feature maps of the block, where each first feature map is of size S×S; and classifiers. Each classifier includes classification layers, each classification layer receives second feature maps having a respective feature dimension. Each classifier is configured to infer partition decisions for sub-blocks of size (αS)×(αS) of the block, wherein α is a power of 2 and α=2, . . . , N/S, by: applying, at some of successive classification layers of the classification layers, a kernel of size 1×1 to reduce the respective feature dimension in half; and outputting by a last layer of the classification layers an output corresponding to a N/(αS)×N/(αS)×1 output map.

公开(公告)号：US11956447B2

公开(公告)日：2024-04-09

申请号：US17601639

申请日：2019-03-21

Applicant: Google LLC

Inventor： Claudionor Coelho ,  Aki Kuusela ,  Joseph Young ,  Shan Li ,  Dake He

IPC: H04N19/147 ,  G06T9/00 ,  H04N19/176 ,  H04N19/96

CPC classification number: H04N19/147 ,  G06T9/002 ,  H04N19/176 ,  H04N19/96

Abstract: An apparatus for encoding an image block includes a processor that presents, to a machine-learning model, the image block, obtains the partition decision for encoding the image block from the model, and encodes the image block using the partition decision. The model is trained to output a partition decision for encoding the image block by using training data for a plurality of training blocks as input, the training data including for a training block, partition decisions for encoding the training block, and, for each partition decision, a rate-distortion value resulting from encoding the training block using the partition decision. The model is trained using a loss function combining a partition loss function based upon a relationship between the partition decisions and respective predicted partitions, and a rate-distortion cost loss function based upon a relationship between the rate-distortion values and respective predicted rate-distortion values.

公开(公告)号：US11310498B2

公开(公告)日：2022-04-19

申请号：US17086591

申请日：2020-11-02

Applicant: GOOGLE LLC

Inventor： Claudionor Coelho ,  Aki Kuusela ,  Shan Li ,  Dake He

IPC: H04N19/119 ,  H04N19/19 ,  H04N19/147 ,  H04N19/176

Abstract: An apparatus for encoding a block of a picture includes a convolutional neural network (CNN) for determining a block partitioning of the block, the block having an N×N size and a smallest partition determined by the CNN being of size S×S. The CNN includes feature extraction layers; a concatenation layer that receives, from the feature extraction layers, first feature maps of the block, where each first feature map of the first feature maps is of the smallest possible partition size S×S of the block; and at least one classifier that is configured to infer partition decisions for sub-blocks of size (αS)×(αS) of the block, where α is a power of 2.

公开(公告)号：US20210051322A1

公开(公告)日：2021-02-18

申请号：US17086591

申请日：2020-11-02

Applicant: GOOGLE LLC

Inventor： Claudionor Coelho ,  Aki Kuusela ,  Shan Li ,  Dake He

IPC: H04N19/119 ,  H04N19/19 ,  H04N19/147 ,  H04N19/176

Abstract: An apparatus for encoding a block of a picture includes a convolutional neural network (CNN) for determining a block partitioning of the block, the block having an N×N size and a smallest partition determined by the CNN being of size S×S. The CNN includes feature extraction layers; a concatenation layer that receives, from the feature extraction layers, first feature maps of the block, where each first feature map of the first feature maps is of the smallest possible partition size S×S of the block; and at least one classifier that is configured to infer partition decisions for sub-blocks of size (αS)×(αS) of the block, where α is a power of 2.