公开(公告)号：US12062150B2

公开(公告)日：2024-08-13

申请号：US17362003

申请日：2021-06-29

申请人： TENCENT AMERICA LLC

发明人： Ding Ding ,  Wei Jiang ,  Wei Wang ,  Shan Liu ,  Xiaozhong Xu

IPC分类号： G06T3/4046 ,  G06N3/045 ,  G06T9/00 ,  H04N19/105 ,  H04N19/117 ,  H04N19/159 ,  H04N19/176 ,  H04N19/60

CPC分类号： G06T3/4046 ,  G06N3/045 ,  G06T9/002 ,  H04N19/105 ,  H04N19/117 ,  H04N19/159 ,  H04N19/176 ,  H04N19/60

摘要： A method of block-wise neural image compression with post filtering is performed by at least one processor of an encoder and includes encoding a block of an input image, using a first neural network, wherein the encoded block is decoded by a decoder using a second neural network to generate a reconstructed block, and performing intra-prediction on the reconstructed block, using a third neural network, to generate a predicted block. The method further includes determining a difference between the block of the input image and the generated predicted block, to generate a prediction residual, encoding the generated prediction residual, using a fourth neural network, wherein the encoded prediction residual is decoded by the decoder using a fifth neural network, and adding the decoded prediction residual to the generated predicted block, to generate a recovered predicted block.

公开(公告)号：US11948090B2

公开(公告)日：2024-04-02

申请号：US17096126

申请日：2020-11-12

申请人： Tencent America LLC

发明人： Wei Jiang ,  Wei Wang ,  Shan Liu

IPC分类号： G06N3/084 ,  G06F17/18 ,  G06F18/21 ,  G06F18/213 ,  G06T9/00

CPC分类号： G06N3/084 ,  G06F17/18 ,  G06F18/213 ,  G06F18/2163 ,  G06T9/002

摘要： In the present disclosure, a method for compressing a feature map is provided, where the feature map is generated by passing a first input through a deep neural network (DNN). A respective optimal index order and a respective optimal unifying method are determined for each of super-blocks that are partitioned from the feature map. A selective structured unification (SSU) layer is subsequently determined based on the respective optimal index order and the respective optimal unifying method for each of the super-blocks. The SSU layer is added to the DNN to form an updated DNN, and is configured to perform unification operations on the feature map. Further, a first estimated output is determined, where the first estimated output is generated by passing the first input through the updated DNN.

公开(公告)号：US11871043B2

公开(公告)日：2024-01-09

申请号：US18154245

申请日：2023-01-13

申请人： TENCENT AMERICA LLC

发明人： Wei Wang ,  Wei Jiang ,  Shan Liu

IPC分类号： H04N19/96 ,  G06N3/08 ,  H04N19/119 ,  H04N19/13 ,  H04N19/91

CPC分类号： H04N19/96 ,  G06N3/08 ,  H04N19/119 ,  H04N19/13 ,  H04N19/91

摘要： A method of three-dimensional (3D)-Tree coding for neural network model compression, is performed by at least one processor, and includes reshaping a four-dimensional (4D) parameter tensor of a neural network into a 3D parameter tensor of the neural network, the 3D parameter tensor comprising a convolution kernel size, an input feature size, and an output feature size, partitioning the 3D parameter tensor along a plane that is formed by the input feature size and the output feature size into 3D coding tree units (CTU3Ds), partitioning each of the CTU3Ds into a plurality of 3D coding units (CU3Ds) recursively until a predetermined depth, using a quad-tree, and constructing a 3D tree for each of the plurality of CU3Ds, wherein the 3D tree for each of the plurality of CU3Ds is a 3D-Unitree.

公开(公告)号：US11849118B2

公开(公告)日：2023-12-19

申请号：US17730020

申请日：2022-04-26

申请人： Tencent America LLC

发明人： Ding Ding ,  Wei Jiang ,  Wei Wang ,  Shan Liu

IPC分类号： H04N19/132 ,  H04N19/149 ,  H04N19/91 ,  H04N19/13 ,  H04N19/154

CPC分类号： H04N19/132 ,  H04N19/13 ,  H04N19/149 ,  H04N19/154 ,  H04N19/91

摘要： Aspects of the disclosure provide a method and an apparatus for video encoding. The apparatus includes processing circuitry configured to perform an iterative update of sample values of a plurality of samples in an initial input image. The iterative update includes generating a coded representation of a final input image based on the final input image by an encoding neural network (NN) and at least one training module. The final input image has been updated from the initial input image by a number of iterations of the iterative update. The iterative update includes generating a reconstructed image of the final input image based on the coded representation of the final input image by a decoding NN. One of a rate-distortion loss for the final input image or the number of iterations of the iterative update satisfies a pre-determined condition. An encoded image corresponding to the final input image is generated.

公开(公告)号：US11758168B2

公开(公告)日：2023-09-12

申请号：US17730000

申请日：2022-04-26

申请人： Tencent America LLC

发明人： Ding Ding ,  Wei Jiang ,  Wei Wang ,  Shan Liu

IPC分类号： H04N19/00 ,  H04N19/44 ,  H04N19/149 ,  G06N3/04 ,  H04N19/184 ,  H04N19/13

CPC分类号： H04N19/44 ,  G06N3/04 ,  H04N19/13 ,  H04N19/149 ,  H04N19/184

摘要： Aspects of the disclosure provide methods, apparatuses, and a non-transitory computer-readable storage medium for video encoding and video decoding. An apparatus for video decoding can include processing circuitry. The processing circuitry is configured to decode neural network update information in a coded bitstream for at least one neural network in the video decoder. The at least one neural network is configured with a set of pretrained parameters, and the neural network update information indicates a first modification parameter. The processing circuitry is configured to update the set of pretrained parameters in the at least one neural network in the video decoder based on the first modification parameter. The processing circuitry is configured to decode an encoded image based on the updated at least one neural network.

公开(公告)号：US11594008B2

公开(公告)日：2023-02-28

申请号：US17085212

申请日：2020-10-30

申请人： TENCENT AMERICA LLC

发明人： Wei Wang ,  Wei Jiang ,  Shan Liu

IPC分类号： G06V10/00 ,  G06V10/424 ,  G06F9/38 ,  G06N3/08

摘要： A method of an escape reorder mode for neural network model compression, is performed by at least one processor, and includes determining whether a frequency count of a codebook index included in a predicted codebook is less than a predetermined value, the codebook index corresponding to a neural network. The method further includes, based on the frequency count of the codebook index being determined to be greater than the predetermined value, maintaining the codebook index, and based on the frequency count of the codebook index being determined to be less than the predetermined value, assigning the codebook index to be an escape index of 0 or a predetermined number. The method further includes encoding the codebook index, and transmitting the encoded codebook index.

公开(公告)号：US11582470B2

公开(公告)日：2023-02-14

申请号：US17333319

申请日：2021-05-28

申请人： TENCENT AMERICA LLC

发明人： Wei Jiang ,  Wei Wang ,  Ding Ding ,  Shan Liu

IPC分类号： H04N19/30 ,  H04N19/149 ,  H04N19/159 ,  H04N19/176 ,  G06N3/08

摘要： A method of multi-scale neural image compression with intra-prediction residuals is performed by at least one processor and includes downsampling an input image, generating a current predicted image, based on a previously-recovered predicted image, and generating a prediction residual based on a difference between the downsampled input image and the generated current predicted image. The method further includes encoding the generated prediction residual, decoding the encoded prediction residual, and generating a currently-recovered predicted image based on an addition of the current predicted image and the decoded prediction residual. The method further includes upsampling the currently-recovered predicted image, generating a scale residual based on a difference between the input image and the upsampled currently-recovered predicted image, and encoding the scale residual.

公开(公告)号：US11496775B2

公开(公告)日：2022-11-08

申请号：US17088075

申请日：2020-11-03

申请人： TENCENT AMERICA LLC

发明人： Wei Jiang ,  Wei Wang ,  Shan Liu

IPC分类号： H04N19/96 ,  G06N3/04 ,  G06N3/08

摘要： A method, computer program, and computer system is provided for compressing a neural network model. One or more coding tree units are identified corresponding to a multi-dimensional tensor associated with a neural network. A set of weight coefficients associated with the coding tree units is unified. A model of the neural network is compressed based on the unified set of weight coefficients.

公开(公告)号：US20220230362A1

公开(公告)日：2022-07-21

申请号：US17365371

申请日：2021-07-01

申请人： TENCENT AMERICA LLC

发明人： Wei Jiang ,  Wei Wang ,  Shan Liu

IPC分类号： G06T9/00 ,  G06N3/08

摘要： A method of adaptive neural image compression with rate control by meta-learning includes receiving an input image and a hyperparameter; and encoding the received input image, based on the received hyperparameter, using an encoding neural network, to generate a compressed representation. The encoding includes performing a first shared encoding on the received input image, using a first shared encoding layer having first shared encoding parameters, performing a first adaptive encoding on the received input image, using a first adaptive encoding layer having first adaptive encoding parameters, combining the first shared encoded input image and the first adaptive encoded input image, to generate a first combined output, and performing a second shared encoding on the first combined output, using a second shared encoding layer having second shared encoding parameters.

公开(公告)号：US20220222505A1

公开(公告)日：2022-07-14

申请号：US17500339

申请日：2021-10-13

申请人： TENCENT AMERICA LLC

发明人： Wei Jiang ,  Wei Wang ,  Sheng Lin ,  Shan Liu

IPC分类号： G06N3/04 ,  G06N3/08

摘要： Video processing with a multi-quality loop filter using a multi-task neural network is performed by at least one processor and includes generating a first set of masked weight parameters, based on an input and a plurality of quantization parameter values with a corresponding first set of masks and first plurality of weight parameters, for a first set of shared neural network layers, selecting a second set of task specific neural network layers for the plurality of quantization parameter values with a second plurality of weight parameters, based on the plurality of quantization parameter values, computing an inference output, based on the first set of masked weight parameters and the second plurality of weight parameters, and outputting the computed inference output as an enhanced result.