公开(公告)号：US20250119556A1

公开(公告)日：2025-04-10

申请号：US18889977

申请日：2024-09-19

Applicant: QUALCOMM Incorporated

Inventor： Yun Li ,  Dmytro Rusanovskyy ,  Thomas Alexander Ryder ,  Samuel James Eadie ,  Marta Karczewicz

IPC: H04N19/176 ,  G06T9/00 ,  H04N19/117 ,  H04N19/159 ,  H04N19/82

Abstract: A method of processing video data includes receiving a picture; and filtering a current block of the picture, through a neural network and based on local correlations of proximate samples and distant, non-local correlations of non-proximate samples relative to the current block, to generate a filtered current block. The neural network comprises one or more backbone blocks and one or more transformer blocks. Each of the one or more transformer blocks is associated with a backbone block of the one or more backbone blocks. At least one of the backbone blocks is configured to capture the local correlations, relative to the current block and the proximate samples of the current block, and at least one of the transformer blocks is configured to generate features, based on applying an attention mechanism, that capture the distant, non-local correlations, relative to the current block and the non-proximate samples, in the picture for processing.

公开(公告)号：US20250008134A1

公开(公告)日：2025-01-02

申请号：US18756952

申请日：2024-06-27

Applicant: QUALCOMM Incorporated

Inventor： Dmytro Rusanovskyy ,  Yun Li ,  Marta Karczewicz

IPC: H04N19/42 ,  H04N19/117 ,  H04N19/172 ,  H04N19/176 ,  H04N19/30 ,  H04N19/50 ,  H04N19/82

Abstract: A device for decoding video data determines a block of a picture; applies a neural network (NN)-based filter process to the block to generate a filtered block, wherein to apply the NN-based filter process, the device performs a first feature extraction on pixel data of the block at a first scale to generate a first set of extracted features for the block; and performs a second feature extraction on the pixel data of the block at a second scale to generate a second set of extracted features for the block, wherein the first scale is different than the second scale; and generates the filtered block based on the first set of extracted features and the second set of extracted features.

公开(公告)号：US20240414378A1

公开(公告)日：2024-12-12

申请号：US18738842

申请日：2024-06-10

Applicant: QUALCOMM Incorporated

Inventor： Yun Li ,  Dmytro Rusanovskyy ,  Marta Karczewicz

IPC: H04N19/80 ,  H04N19/172

Abstract: Example techniques for filtering video data are described. An example device for at least one of encoding or decoding video data includes one or more memories configured to store the video data and one or more processors. The one or more processors are configured to receive a picture of video data and reconstruct the picture of video data. The one or more processors are also configured to apply a neural network (NN)-based filter to the reconstructed picture of video data. The NN-based filter includes a unified filter. The unified filter includes a head block, a transition block, one or more backbone blocks, and a tail block. At least one of the head block, the transition block, the one or more backbone blocks, or the tail block includes a Canonical Polyadic (CP) decomposition with separable convolution.

公开(公告)号：US20240422361A1

公开(公告)日：2024-12-19

申请号：US18742216

申请日：2024-06-13

Applicant: QUALCOMM Incorporated

Inventor： Dmytro Rusanovskyy ,  Yun Li ,  Marta Karczewicz

IPC: H04N19/86 ,  H04N19/117 ,  H04N19/167 ,  H04N19/172 ,  H04N19/44

Abstract: An example device for decoding video data includes: a memory configured to store video data; and a processing system comprising one or more processors implemented in circuitry, the processing system being configured to: decode at least a portion of a picture of video data; combine two or more sets of supplementary data for the at least portion of the picture into a single set of supplementary data; and execute a neural network filter, using the at least portion of the picture and the single set of supplementary data as inputs to the neural network filter, to filter the at least portion of the picture.

公开(公告)号：US20240414377A1

公开(公告)日：2024-12-12

申请号：US18738612

申请日：2024-06-10

Applicant: QUALCOMM Incorporated

Inventor： Yun Li ,  Dmytro Rusanovskyy ,  Marta Karczewicz

IPC: H04N19/80 ,  H04N19/176

Abstract: A device for decoding video data receives a picture of video data; reconstructs a block of the picture of video data to generate a reconstructed block; applies a neural network (NN)-based filter process to the reconstructed block to generate a filtered block, wherein the NN-based filter process includes a first backbone block process followed by a second backbone block process, wherein the first backbone block process comprises a first M×N convolution followed by a first N×M convolution, and the second backbone block process comprises a second N×M convolution followed by a second M×N convolution, wherein N and M are different integer values; determines a decoded block of video data based on the filtered block; and outputs a decoded version of the picture, wherein the decoded version of the picture comprises the decoded block of video data.

公开(公告)号：US20180174623A1

公开(公告)日：2018-06-21

申请号：US15381587

申请日：2016-12-16

Applicant: QUALCOMM Incorporated

Inventor： Fei Xu ,  Rakesh Vattikonda ,  Dina McKinney ,  Zhen Chen ,  Yun Li ,  Zhenbiao Ma ,  De Lu

IPC: G11C7/10 ,  G06F3/06

CPC classification number: G11C7/1012 ,  G11C7/222

Abstract: An apparatus and method are disclosed for transferring data from a first core to a second core of an integrated circuit (IC). The first core includes first and second memory blocks (e.g., first and second portions of a first-in-first-out (FIFO) memory coupled to first and second pre-multiplexers, respectively). The second core includes a multiplexer including first and second inputs coupled to the first and second memory blocks, respectively. Additionally, the second core includes a read controller configured to generate a first read control signal to cause the first and second memory blocks to transfer data to the first and second inputs of the multiplexer, respectively; and generate a second read control signal to cause the multiplexer to transfer data from the first and inputs to an output of the multiplexer.

公开(公告)号：US20250119592A1

公开(公告)日：2025-04-10

申请号：US18888423

申请日：2024-09-18

Applicant: QUALCOMM Incorporated

Inventor： Thomas Alexander Ryder ,  Dmytro Rusanovskyy ,  Samuel James Eadie ,  Yun Li ,  Marta Karczewicz

IPC: H04N19/82 ,  H04N19/117 ,  H04N19/132 ,  H04N19/159 ,  H04N19/176

Abstract: Methods and devices for decoding video data are described. An example method includes in-loop filtering a current block of the video data using a neural network-based in-loop filter to generate an in-loop filtered current block, wherein the neural network-based in-loop filter is trained using an architecture comprising a U-Net architecture comprising one or more residual blocks and one or more transform blocks; and outputting the in-loop filtered current block.

公开(公告)号：US20250097474A1

公开(公告)日：2025-03-20

申请号：US18883696

申请日：2024-09-12

Applicant: QUALCOMM Incorporated

Inventor： Yun Li ,  Dmytro Rusanovskyy ,  Marta Karczewicz

IPC: H04N19/80 ,  H04N19/117 ,  H04N19/124 ,  H04N19/172 ,  H04N19/176 ,  H04N19/42 ,  H04N19/463

Abstract: A device for decoding video data is configured to determine first neural network (NN) weights for a layer of a convolutional neural network (CNN) filter; derive a quantization value for the layer of the CNN filter based on values of the first NN weights; convert the first NN weights to second NN weights based on the quantization value; and filter a block of the video data using the second NN weights.

公开(公告)号：US20240348837A1

公开(公告)日：2024-10-17

申请号：US18631748

申请日：2024-04-10

Applicant: QUALCOMM Incorporated

Inventor： Dmytro Rusanovskyy ,  Yun Li ,  Samuel James Eadie ,  Marta Karczewicz

IPC: H04N19/82 ,  H04N19/117 ,  H04N19/172 ,  H04N19/176

CPC classification number: H04N19/82 ,  H04N19/117 ,  H04N19/172 ,  H04N19/176

Abstract: A device for decoding video data receives a picture of video data; reconstructs a block of the picture of video data to generate a reconstructed block; and performs a neural network (NN)-based filter process on the reconstructed block to generate a filtered block, wherein the NN-based filter process includes performing a plurality of separable convolutions in parallel with a point-wise input convolution.

公开(公告)号：US20240283925A1

公开(公告)日：2024-08-22

申请号：US18442955

申请日：2024-02-15

Applicant: QUALCOMM Incorporated

Inventor： Dmytro Rusanovskyy ,  Samuel James Eadie ,  Yun Li ,  Marta Karczewicz

IPC: H04N19/117 ,  H04N19/105 ,  H04N19/176

CPC classification number: H04N19/117 ,  H04N19/105 ,  H04N19/176

Abstract: A video coder is configured to perform a neural network (NN)-based filter process on reconstructed blocks of vide data. In one example, a video coder may receive a picture of video data, and reconstruct a block of the picture of video data to generate a reconstructed block. The video coder may perform the NN-based filter process on the reconstructed block to generate a filtered block, wherein the NN-based filter process includes performing a plurality separable convolutions to approximate a multi-dimensional convolution.