公开(公告)号：US11432008B2

公开(公告)日：2022-08-30

申请号：US17341825

申请日：2021-06-08

Applicant: Sony Interactive Entertainment Inc.

Inventor： Jason N. Wang ,  Eric Hsuming Chen

IPC: H04N19/593 ,  H04N19/436 ,  H04N19/563 ,  H04N19/107 ,  H04N19/523 ,  H04L65/70

Abstract: Training a video decoder system may include masking one of at least two sets of video encoding parameters with invalid values to generate an invalid set. The at least two sets of video encoding parameters are provided to one or more neural networks to train them to predict valid values that correspond to values of the invalid set using an iterative training algorithm. Encoding parameters to encode are determined based on a prediction error of the one or more neural networks. Encoding parameters which are determined to be accurately predicted are dropped from the encoded data. A new video stream is encoded without the dropped encoding parameters.

公开(公告)号：US20210297695A1

公开(公告)日：2021-09-23

申请号：US17341825

申请日：2021-06-08

Applicant: Sony Interactive Entertainment Inc.

Inventor： Jason N. Wang ,  Eric Hsuming Chen

IPC: H04N19/593 ,  H04N19/436 ,  H04L29/06 ,  H04N19/563 ,  H04N19/107 ,  H04N19/523

Abstract: Training a video decoder system may include masking one of at least two sets of video encoding parameters with invalid values to generate an invalid set. The at least two sets of video encoding parameters are provided to one or more neural networks to train them to predict valid values that correspond to values of the invalid set using an iterative training algorithm. Encoding parameters to encode are determined based on a prediction error of the one or more neural networks. Encoding parameters which are determined to be accurately predicted are dropped from the encoded data. A new video stream is encoded without the dropped encoding parameters.

公开(公告)号：US11164339B2

公开(公告)日：2021-11-02

申请号：US16721733

申请日：2019-12-19

Applicant: Sony Interactive Entertainment Inc.

Inventor： Rathish Krishnan ,  Jason N. Wang

IPC: G06K9/00 ,  G06T9/00 ,  G06T3/40 ,  G06T5/20 ,  H04N19/176 ,  H04N19/167 ,  H04N19/132 ,  G06T5/00

Abstract: A method, system and computer readable instructions for video encoding comprising, determining one or more region of interest (ROI) parameters for pictures in a picture stream and a temporal down sampling interval. One or more areas outside the ROI in a picture in the picture stream are temporally down sampled according to the interval. The resulting temporally down sampled picture is then encoded and the encoded temporally down-sampled picture is transmitted. Additionally, a picture encoded in this way in an encoded picture stream may be decoded and areas outside an ROI of the picture may be temporally up sampled. The temporally up sampled areas outside the ROI are inserted into the decoded encoded picture stream.

公开(公告)号：US11032569B2

公开(公告)日：2021-06-08

申请号：US16684279

申请日：2019-11-14

Applicant: Sony Interactive Entertainment Inc.

Inventor： Jason N. Wang ,  Eric Hsuming Chen

IPC: H04N19/593 ,  H04N19/436 ,  H04N19/563 ,  H04N19/107 ,  H04N19/523 ,  H04L29/06

Abstract: Training a video decoder system may include generating at least two valid sets of video encoding parameters and masking one of the sets with invalid values to generate an invalid set. The valid sets of video encoding parameters may be provided to one or more neural networks to train them to predict valid values that correspond to values of the invalid set using an iterative training algorithm. A prediction error of the predicted valid values is determined from the results of the training of the neural networks and the valid video encoding parameters. The prediction error is inserted into encoding data and encoded parameters determined to be accurately predicted with the addition of the prediction error are dropped. A new video stream is encoded with the predication error and without the dropped encoding parameters.

公开(公告)号：US09740886B2

公开(公告)日：2017-08-22

申请号：US14192782

申请日：2014-02-27

Applicant: Sony Interactive Entertainment Inc.

Inventor： Jason N. Wang ,  Cheng Huang

IPC: G06F21/76 ,  G06F12/14 ,  G06F21/55

CPC classification number: G06F21/76 ,  G06F12/14 ,  G06F21/556

Abstract: A software security layer may be used to protect a system against exploitation of a hardware encoder accelerator by malicious data embedded in the one or more frames of encoded digital streaming data. It is emphasized that this abstract is provided to comply with the rules requiring an abstract that will allow a searcher or other reader to quickly ascertain the subject matter of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims.

公开(公告)号：US20210266571A1

公开(公告)日：2021-08-26

申请号：US17313882

申请日：2021-05-06

Applicant: Sony Interactive Entertainment Inc.

Inventor： Eric Hsuming Chen ,  Hung-Ju Lee ,  Jason N. Wang ,  Rathish Krishnan ,  Deepali Arya

IPC: H04N19/167 ,  G06T7/73 ,  H04N19/59 ,  H04N19/17 ,  H04N19/132 ,  H04N19/119 ,  H04N19/172 ,  G06T7/11 ,  G06T3/40 ,  G06T11/60

Abstract: Video stream data is selectively scaled so that sections within regions of interest (ROI) maintain high resolution while areas not within the region of interest are down-scaled to reduce bandwidth cost of transmission. A low compression encoder compresses sections of a video frame corresponding to one or more ROI without motion search or prediction mode decision to generate low-compression section data. The video frame is downscaled and a high compression encoder compresses the resulting downscaled video frame with prediction mode decision to generate high-compression frame data.

公开(公告)号：US20210142520A1

公开(公告)日：2021-05-13

申请号：US16721733

申请日：2019-12-19

Applicant: Sony Interactive Entertainment Inc.

Inventor： Rathish Krishnan ,  Jason N. Wang

IPC: G06T9/00 ,  G06T3/40 ,  G06T5/20 ,  G06T5/00 ,  H04N19/167 ,  H04N19/132 ,  H04N19/176

Abstract: A method, system and computer readable instructions for video encoding comprising, determining one or more region of interest (ROI) parameters for pictures in a picture stream and a temporal down sampling interval. One or more areas outside the ROI in a picture in the picture stream are temporally down sampled according to the interval. The resulting temporally down sampled picture is then encoded and the encoded temporally down-sampled picture is transmitted. Additionally, a picture encoded in this way in an encoded picture stream may be decoded and areas outside an ROI of the picture may be temporally up sampled. The temporally up sampled areas outside the ROI are inserted into the decoded encoded picture stream.

公开(公告)号：US20190387252A1

公开(公告)日：2019-12-19

申请号：US16012570

申请日：2018-06-19

Applicant: Sony Interactive Entertainment Inc.

Inventor： Jason N. Wang ,  Eric Hsuming Chen

IPC: H04N19/593 ,  H04N19/436 ,  H04N19/523 ,  H04N19/563 ,  H04N19/107 ,  H04L29/06

Abstract: Training a video decoder system comprising, generating at least two sets of video encoding parameters wherein the at least two sets of video encoding parameters are valid, masking a set of the at least two sets of encoding parameters with invalid values to generate an invalid set of video encoding parameters, providing a set of the at least two sets of video encoding parameters to a neural network, training the neural network to predict valid video encoding parameter values for the invalid set using an iterative training algorithm, determining which encoding parameters need to be encoded based on analysis of a prediction error of the trained recurrent neural network, dropping the encoding parameters from the encoded data which are determined to be accurately predicted by the trained recurrent neural network, encoding a new video stream without the dropped encoding parameters. A coder and decoder system with neural network is also disclosed.

公开(公告)号：US10349069B2

公开(公告)日：2019-07-09

申请号：US13710794

申请日：2012-12-11

Applicant: Sony Interactive Entertainment Inc.

Inventor： Jason N. Wang ,  Hung-Ju Lee

IPC: H04B1/66 ,  H04N19/42 ,  H04N19/156

Abstract: In aspects of the present disclosure, a software encoder augments a hardware encoder by implementing portions of a video encoding task that are not supported by the hardware encoder while the hardware encoder implements other portions of the encoding task. The use of a software encoder to augment a hardware encoder in this manner can extend the useful life of a hardware encoder, allow the system to adapt to changes in video coding standards, and in some cases improve performance of encoding implemented by hardware or software alone. It is emphasized that this abstract is provided to comply with the rules requiring an abstract that will allow a searcher or other reader to quickly ascertain the subject matter of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims.

公开(公告)号：US11546610B2

公开(公告)日：2023-01-03

申请号：US17313882

申请日：2021-05-06

Applicant: Sony Interactive Entertainment Inc.

Inventor： Eric Hsuming Chen ,  Hung-Ju Lee ,  Jason N. Wang ,  Rathish Krishnan ,  Deepali Arya

IPC: H04N19/167 ,  H04N19/119 ,  H04N19/172 ,  G06T3/40 ,  G06T7/11 ,  G06T11/60 ,  G06T7/73 ,  H04N19/59 ,  H04N19/17 ,  H04N19/132

Abstract: Video stream data is selectively scaled so that sections within regions of interest (ROI) maintain high resolution while areas not within the region of interest are down-scaled to reduce bandwidth cost of transmission. A low compression encoder compresses sections of a video frame corresponding to one or more ROI without motion search or prediction mode decision to generate low-compression section data. The video frame is downscaled and a high compression encoder compresses the resulting downscaled video frame with prediction mode decision to generate high-compression frame data.