公开(公告)号：US20250008132A1

公开(公告)日：2025-01-02

申请号：US18755150

申请日：2024-06-26

Applicant: NEC Laboratories America, Inc.

Inventor： Biplob Debnath ,  Deep Patel ,  Srimat Chakradhar ,  Christoph Reich

IPC: H04N19/33 ,  H04N19/124 ,  H04N19/176 ,  H04N19/186 ,  H04N19/625

Abstract: Systems and methods are provided for encoding and decoding images using differentiable JPEG compression, including converting images from RGB color space to YCbCr color space to obtain a luminance and chrominance channels, and applying chroma subsampling to the chrominance channels to reduce resolution. The YCbCr image is divided into pixel blocks and a DCT is performed on the pixel blocks to obtain DCT coefficients. DCT coefficients are quantized using a scaled quantization table to reduce precision, and quantized DCT coefficients are encoded using lossless entropy coding, forming a compressed JPEG file decoded by reversing the lossless entropy coding to obtain quantized DCT coefficients, which are dequantized using the scaled quantization table to restore the precision. The dequantized DCT coefficients are converted back to a spatial domain using an IDCT, the chrominance channels are upsampled to original resolution, and the YCbCr image is converted back to the RGB color space.

公开(公告)号：US20240275996A1

公开(公告)日：2024-08-15

申请号：US18439291

申请日：2024-02-12

Applicant: NEC Laboratories America, Inc.

Inventor： Biplob Debnath ,  Deep Patel ,  Srimat Chakradhar ,  Oliver Po ,  Christoph Reich

IPC: H04N19/42 ,  G06N20/00 ,  H04N7/18 ,  H04N19/119 ,  H04N19/124 ,  H04N19/14 ,  H04N19/154 ,  H04N19/156 ,  H04N19/172 ,  H04N19/176 ,  H04N19/177 ,  H04N19/463 ,  H04N19/61

CPC classification number: H04N19/42 ,  G06N20/00 ,  H04N7/183 ,  H04N19/119 ,  H04N19/124 ,  H04N19/14 ,  H04N19/154 ,  H04N19/156 ,  H04N19/172 ,  H04N19/176 ,  H04N19/177 ,  H04N19/463 ,  H04N19/61

Abstract: Systems and methods are provided for optimizing video compression using end-to-end learning, including capturing, using an edge device, raw video frames from a video clip and determining maximum network bandwidth. Predicting, using a control network implemented on the edge device, optimal codec parameters, based on dynamic network conditions and content of the video clip, encoding, using a differentiable surrogate model of a video codec, the video clip using the predicted codec parameters and to propagate gradients from a server-side vision model to adjust the codec parameters. Decoding, using a server, the video clip and analyzing the video clip with a deep vision model located on the server, transmitting, using a feedback mechanism, analysis from the deep vision model back to the control network to facilitate end-to-end training of the system. Adjusting the encoding parameters based on the analysis from the deep vision model received from the feedback mechanism.

公开(公告)号：US20240275983A1

公开(公告)日：2024-08-15

申请号：US18439341

申请日：2024-02-12

Applicant: NEC Laboratories America, Inc.

Inventor： Biplob Debnath ,  Christoph Reich ,  Deep Patel ,  Srimat Chakradhar

IPC: H04N19/146 ,  G06V20/40 ,  G06V20/58 ,  H04N19/124 ,  H04N19/154

CPC classification number: H04N19/146 ,  G06V20/49 ,  G06V20/58 ,  H04N19/124 ,  H04N19/154

Abstract: Systems and methods are provided for optimizing video compression for remote vehicle control, including capturing, capturing video and sensor data from a vehicle using a plurality of sensors and high-resolution cameras, analyzing the captured video to identify critical regions within frames of the video using an attention-based module. Current network bandwidth is assessed and future bandwidth availability is predicted. Video compression parameters are predicted based on an analysis of the video and an assessment of the current network bandwidth using a control network, and the video is compressed based on the predicted parameters with an adaptive video compression module. The compressed video and sensor data is transmitted to a remote-control center, and received video and sensor data is decoded at the remote-control center. The vehicle is autonomously or remotely controlled from the remote-control center based on the decoded video and sensor data.