公开(公告)号：US20220360778A1

公开(公告)日：2022-11-10

申请号：US17621476

申请日：2020-07-02

Applicant: VID SCALE, INC.

Inventor： Hua Yang ,  Duanshun Li ,  Dong Tian ,  Yuwen He

IPC: H04N19/119 ,  H04N19/96 ,  H04N19/105 ,  H04N19/172

Abstract: A method of encoding or decoding a video comprising a current picture, a first reference picture, and a weight tensor associated with a trained neural network (NN) model are provided. The method includes generating any number of kernel tensors, input channels and output channels associated with the weight tensor, each kernel tensor being associated with any of: a layer type, an input signal type, and a tree partition type, and each kernel tensor including weight coefficients, generating, for each of the any number of kernel tensors, tree partitions for any of a coding tree unit (CTU), a coding unit (CU), a prediction unit (PU), and a transform unit (TU) according to respective tree partition types associated with each of the any number of kernel tensors, and generating a compressed representation of the trained NN model by compressing and coding the any number of kernel tensors.

公开(公告)号：US20220261616A1

公开(公告)日：2022-08-18

申请号：US17622954

申请日：2020-07-01

Applicant: VID SCALE, INC.

Inventor： Duanshun Li ,  Dong Tian ,  Hua Yang ,  Yuwen He

IPC: G06N3/04

Abstract: Systems, methods, and instrumentalities are disclosed for clustering-based quantization for neural network (NN) compression. A distribution of weights in weight tensors in NN layers may be analyzed to identify cluster outliers. Cluster inliers may be coded from cluster outliers, for example, using scalar and/or vector quantization. Weight-rearrangement may rearrange weights for higher dimensional weight tensors into lower dimensional matrices. For example, weight rearrangement may flatten a convolutional kernel into a vector. Correlation between kernels may be preserved, for example, by treating a filter or kernels across a channel as a point. A tensor may be split into multiple subspaces, for example, along an input and/or an output channel. Predictive coding may be performed for a current block of weights or weight matrix based on a reshaped or previously coded block or matrix. Arrangement, inlier, outlier, and/or prediction information may be signaled to a decoder for reconstruction of a compressed NN.