公开(公告)号：US20230199179A1

公开(公告)日：2023-06-22

申请号：US18173286

申请日：2023-02-23

Applicant: GOOGLE LLC

Inventor： Debargha Mukherjee ,  Urvang Joshi ,  Yue Chen ,  Sarah Parker

IPC: H04N19/107 ,  H04N19/105 ,  H04N19/154 ,  H04N19/184 ,  G06N20/00 ,  G06N3/08 ,  G06N3/045

CPC classification number: H04N19/107 ,  G06N3/08 ,  G06N3/045 ,  G06N20/00 ,  H04N19/105 ,  H04N19/154 ,  H04N19/184

Abstract: Video coding may include generating, by a processor, a decoded frame by decoding a current frame from an encoded bitstream and outputting a reconstructed frame based on the decoded frame. Decoding includes identifying a current encoded block from the current frame, identifying a prediction coding model for the current block, wherein the prediction coding model is a machine learning prediction coding model from a plurality of machine learning prediction coding models, identifying reference values for decoding the current block based on the prediction coding model, obtaining prediction values based on the prediction coding model and the reference values, generating a decoded block corresponding to the current encoded block based on the prediction values, and including the decoded block in the decoded frame.

公开(公告)号：US11601644B2

公开(公告)日：2023-03-07

申请号：US16295176

申请日：2019-03-07

Applicant: GOOGLE LLC

Inventor： Debargha Mukherjee ,  Urvang Joshi ,  Yue Chen ,  Sarah Parker

IPC: H04N19/107 ,  H04N19/105 ,  H04N19/154 ,  H04N19/184 ,  G06N20/00 ,  G06N3/04 ,  G06N3/08

Abstract: Video coding may include generating, by a processor, a decoded frame by decoding a current frame from an encoded bitstream and outputting a reconstructed frame based on the decoded frame. Decoding includes identifying a current encoded block from the current frame, identifying a prediction coding model for the current block, wherein the prediction coding model is a machine learning prediction coding model from a plurality of machine learning prediction coding models, identifying reference values for decoding the current block based on the prediction coding model, obtaining prediction values based on the prediction coding model and the reference values, generating a decoded block corresponding to the current encoded block based on the prediction values, and including the decoded block in the decoded frame.

公开(公告)号：US20220207654A1

公开(公告)日：2022-06-30

申请号：US17698116

申请日：2022-03-18

Applicant: GOOGLE LLC

Inventor： Debargha Mukherjee ,  Urvang Joshi ,  Yue Chen ,  Sarah Parker

IPC: G06T5/00 ,  G06T5/50 ,  G06N3/08 ,  G06N3/04 ,  G06N20/20 ,  G06T3/40 ,  H04N19/70 ,  H04N19/176 ,  G06K9/62 ,  G06T9/00

Abstract: Guided restoration is used to restore video data degraded from a video frame. The video frame is divided into restoration units (RUs) which each correspond to one or more blocks of the video frame. Restoration schemes are selected for each RU. The restoration schemes may indicate to use one of a plurality of neural networks trained for the guided restoration. Alternatively, the restoration schemes may indicate to use a neural network and a filter-based restoration tool. The video frame is then restored by processing each RU according to the respective selected restoration scheme. During encoding, the restored video frame is encoded to an output bitstream, and the use of the selected restoration schemes may be signaled within the output bitstream. During decoding, the restored video frame is output to an output video stream.

公开(公告)号：US11870993B2

公开(公告)日：2024-01-09

申请号：US17359767

申请日：2021-06-28

Applicant: GOOGLE LLC

Inventor： Urvang Joshi ,  Debargha Mukherjee

IPC: H04N19/122 ,  H04N19/176 ,  H04N19/82 ,  H04N19/18 ,  H04N19/159 ,  H04N19/124

CPC classification number: H04N19/122 ,  H04N19/124 ,  H04N19/159 ,  H04N19/176 ,  H04N19/18 ,  H04N19/82

Abstract: Improved transforms are used to encode and decode large video and image blocks. During encoding, a prediction residual block having a large size (e.g., larger than 32×32) is generated. The pixel values of the prediction residual block are transformed to produce transform coefficients. After determining that the transform coefficients exceed a threshold cardinality representative of a maximum transform block size (e.g., 32×32), a number of the transform coefficients are discarded such that a remaining number of transform coefficients does not exceed the threshold cardinality. A transform block is then generated using the remaining number. During decoding, after determining that the transform coefficients exceed the threshold cardinality, a number of new coefficients are added to the transform coefficients such that a total number of transform coefficients exceeds the threshold cardinality. The transform coefficients are then inverse transformed into a prediction residual block having a large size.

公开(公告)号：US11558631B2

公开(公告)日：2023-01-17

申请号：US16836078

申请日：2020-03-31

Applicant: GOOGLE LLC

Inventor： Urvang Joshi ,  Debargha Mukherjee ,  Andrew Simpson

IPC: H04N19/117 ,  H04N19/176 ,  H04N19/50 ,  H04N7/01 ,  H04N19/184 ,  H04N19/61

Abstract: A super-resolution coding mode is described. Encoded image can be decoded by decoding, from an encoded bitstream, a flag indicating whether an image was encoded using the super-resolution mode. The image is encoded at a first resolution. Responsive to the flag indicating that the image was encoded using the super-resolution mode, bits indicating an amount of scaling of the image are decoded. The image is decoded from the encoded bitstream to obtain a reconstructed image at the first resolution, and the reconstructed image is upscaled to a second resolution using the amount of scaling to obtain an upscaled reconstructed image. The second resolution is higher than the first resolution. Loop restoration filtering is applied to the upscaled reconstructed image using loop restoration parameters to obtain a loop restored image at the second resolution.

公开(公告)号：US10771783B2

公开(公告)日：2020-09-08

申请号：US16004929

申请日：2018-06-11

Applicant: GOOGLE LLC

Inventor： Urvang Joshi ,  Debargha Mukherjee

IPC: H04N19/122 ,  H04N19/176 ,  H04N19/82 ,  H04N19/18 ,  H04N19/159 ,  H04N19/124

Abstract: Improved transforms are used to encode and decode large video and image blocks. During encoding, a prediction residual block having a large size (e.g., larger than 32×32) is generated. The pixel values of the prediction residual block are transformed to produce transform coefficients. After determining that the transform coefficients exceed a threshold cardinality representative of a maximum transform block size (e.g., 32×32), a number of the transform coefficients are discarded such that a remaining number of transform coefficients does not exceed the threshold cardinality. A transform block is then generated using the remaining number. During decoding, after determining that the transform coefficients exceed the threshold cardinality, a number of new coefficients are added to the transform coefficients such that a total number of transform coefficients exceeds the threshold cardinality. The transform coefficients are then inverse transformed into a prediction residual block having a large size.

公开(公告)号：US20200186809A1

公开(公告)日：2020-06-11

申请号：US16516784

申请日：2019-07-19

Applicant: GOOGLE LLC

Inventor： Debargha Mukherjee ,  Urvang Joshi ,  Yue Chen ,  Sarah Parker

IPC: H04N19/147 ,  G06N3/04 ,  H04N19/59 ,  H04N19/184

Abstract: A hybrid apparatus for coding a video stream includes a first encoder. The first encoder includes a neural network having at least one hidden layer, and the neural network receives source data from the video stream at a first hidden layer of the at least one hidden layer, receives side information correlated with the source data at the first hidden layer, and generates guided information using the source data and the side information. The first encoder outputs the guided information and the side information for a decoder to reconstruct the source data.

公开(公告)号：US20190335199A1

公开(公告)日：2019-10-31

申请号：US15966005

申请日：2018-04-30

Applicant: GOOGLE LLC

Inventor： Urvang Joshi ,  Debargha Mukherjee

IPC: H04N19/593 ,  H04N19/59 ,  H04N19/61 ,  H04N19/176

Abstract: Coding a block of a video frame using an intra-prediction mode is disclosed. A method includes selecting first neighboring pixels, generating second pixels for use along a second edge that is opposite the first edge of the block, and generating a prediction block that includes predicted pixels. The first neighboring pixels are peripheral to the block along a first edge of the block. The second pixels are generated using third neighboring pixels that are peripheral to a third edge of the block. The third edge is different from the first edge and the second edge. Generating the prediction block includes interpolating, using a first interpolation, the first neighboring pixels and the second pixels to obtain the predicted pixels.

公开(公告)号：US12075081B2

公开(公告)日：2024-08-27

申请号：US18155224

申请日：2023-01-17

Applicant: Google LLC

Inventor： Urvang Joshi ,  Debargha Mukherjee ,  Andrew Simpson

IPC: H04N19/50 ,  H04N7/01 ,  H04N19/117 ,  H04N19/176 ,  H04N19/184 ,  H04N19/61

CPC classification number: H04N19/50 ,  H04N7/0117 ,  H04N19/117 ,  H04N19/176 ,  H04N19/184 ,  H04N19/619

Abstract: A super-resolution coding mode is described. An encoded image can be decoded from an encoded bitstream stored on a non-transitory computer-readable storage medium. A flag can indicate whether an image was encoded using the super-resolution mode at a first resolution. Responsive to the flag indicating that the image was encoded using the super-resolution mode, bits indicating an amount of scaling of the image are included. The image is decoded from the encoded bitstream to obtain a reconstructed image at the first resolution, and the reconstructed image is upscaled to a second resolution using the amount of scaling to obtain an upscaled reconstructed image. The second resolution is higher than the first resolution. Loop restoration parameters within the bitstream can used for look restoration filtering of the upscaled reconstructed image to obtain a loop restored image at the second resolution.

公开(公告)号：US11282172B2

公开(公告)日：2022-03-22

申请号：US16515226

申请日：2019-07-18

Applicant: GOOGLE LLC

Inventor： Debargha Mukherjee ,  Urvang Joshi ,  Yue Chen ,  Sarah Parker

IPC: G06T9/00 ,  G06T5/00 ,  G06T5/50 ,  G06N3/08 ,  G06N3/04 ,  H04N19/70 ,  H04N19/176 ,  G06N20/20 ,  G06T3/40 ,  G06K9/62

Abstract: Guided restoration is used to restore video data degraded from a video frame. The video frame is divided into restoration units (RUs) which each correspond to one or more blocks of the video frame. Restoration schemes are selected for each RU. The restoration schemes may indicate to use one of a plurality of neural networks trained for the guided restoration. Alternatively, the restoration schemes may indicate to use a neural network and a filter-based restoration tool. The video frame is then restored by processing each RU according to the respective selected restoration scheme. During encoding, the restored video frame is encoded to an output bitstream, and the use of the selected restoration schemes may be signaled within the output bitstream. During decoding, the restored video frame is output to an output video stream.