公开(公告)号：US11330296B2

公开(公告)日：2022-05-10

申请号：US17020750

申请日：2020-09-14

Applicant: Apple Inc.

Inventor： Guy Côté ,  Athanasios Leontaris ,  Muge Wang

IPC: H04N19/30 ,  H04N19/61 ,  H04N19/187 ,  H04N19/50 ,  H04N19/59 ,  H04N19/103 ,  H04N19/182 ,  H04N19/517 ,  H04N19/136

Abstract: Systems and methods for improving operational efficiency of a video encoding system used to encode image data are provided. In embodiments, a video encoding system includes image processing circuitry configured to receive source image data and derive full-resolution image data and low-resolution image data from the source image data. The video encoding system also includes a low resolution pipeline configured to receive the low-resolution image data and determine one or more low resolution inter prediction modes based on the low-resolution image data. Furthermore, the video encoding system includes a main pipeline configured to encode the full-resolution image data based on the one or more low resolution inter prediction modes.

公开(公告)号：US10685421B1

公开(公告)日：2020-06-16

申请号：US16664120

申请日：2019-10-25

Applicant: Apple Inc.

Inventor： Sung Hee Park ,  Muge Wang ,  Junji Sugisawa

IPC: G06F17/15 ,  G06T1/20 ,  G06K9/46 ,  G06K9/00 ,  G06N3/04 ,  G06T5/20 ,  G06T5/00 ,  G06N3/08 ,  G06N3/063

Abstract: Embodiments relate to a configurable convolution engine that receives configuration information to perform convolution and other deep machine learning operations on streaming input data of various formats. The convolution engine may include two convolution circuits that each generate a stream of values by applying convolution kernels to input data. The stream of values may each define one or more channels of image data. A channel merge circuit combines the streams of values from each convolution circuit in accordance with a selected mode of operation. In one mode, the first and second streams from the convolution circuits are merged into an output stream having the combined channels of the first and second streams in an interleaved manner. In another mode, the first stream from the first convolution circuit is fed into the input of the second convolution circuit.

公开(公告)号：US20200077066A1

公开(公告)日：2020-03-05

申请号：US16116838

申请日：2018-08-29

Applicant: Apple Inc.

Inventor： Frederic Cao ,  Touraj Tajbakhsh ,  Muge Wang

IPC: H04N9/64 ,  H04N5/232 ,  H04N9/73

Abstract: Embodiments relate to generation of hue maps for highlight recovery of an input image. An image having a plurality of color channels is obtained at a first resolution lower than a resolution of the input image. A hue for each color channel for each pixel is determined, using a pixel value for that color channel and pixel values for the plurality of color channels in the first image. Weights are determined for each pixel for each color channel, based on hues for the pixel and pixel values for the pixel in the first image. A plurality of candidate hue maps are generated, based on the weights and pixel values in the first image in a patch surrounding the pixel for the plurality of color channels.

公开(公告)号：US20200057789A1

公开(公告)日：2020-02-20

申请号：US16664096

申请日：2019-10-25

Applicant: Apple Inc.

Inventor： Suk Hwan Lim ,  Junji Sugisawa ,  Muge Wang

IPC: G06F17/15 ,  G06T5/20 ,  G06T5/00

Abstract: Embodiments of the present disclosure relate to a configurable convolution engine that receives configuration information to perform convolution or its variant operations on streaming input data of various formats. To process streaming input data, input data of multiple channels are received and stored in an input buffer circuit in an interleaved manner. Data values of the interleaved input data are retrieved and forwarded to multiplier circuits where multiplication with a corresponding filter element of a kernel is performed. Varying number of kernels with different sizes and sparsity can also be used for the convolution operations.

公开(公告)号：US20200051256A1

公开(公告)日：2020-02-13

申请号：US16100780

申请日：2018-08-10

Applicant: Apple Inc.

Inventor： Muge Wang ,  Junji Sugisawa

IPC: G06T7/32 ,  G06T7/00 ,  G06T7/20 ,  G06T5/00 ,  G06K9/62 ,  G06T7/33

Abstract: Embodiments relate to a normalized cross correlation (NCC) circuit that can perform a normalized cross correlation between input patch data and kernel data. An interface circuit of an image signal processor receives input patch data from a source. Input patch data is data that represents a portion of a frame of image data from the source. The NCC circuit includes a filtering circuit and a normalization circuit. The filtering circuit receives the input patch data from the interface circuit and performs a convolution on the received input patch data or processed patch data derived from the input patch data with kernel data to produce convolution output data. The normalization circuit computes a normalized score output based on the convolution output data and the kernel data. The normalized score output includes normalization scores for each location of the convolution output data.

公开(公告)号：US20180315154A1

公开(公告)日：2018-11-01

申请号：US15499524

申请日：2017-04-27

Applicant: Apple Inc.

Inventor： Sung Hee Park ,  Muge Wang

IPC: G06T1/20 ,  G06T5/00 ,  G06T5/20

CPC classification number: G06T1/20 ,  G06F17/153 ,  G06K9/00973 ,  G06K9/4642 ,  G06K9/627 ,  G06N3/063 ,  G06T5/001 ,  G06T5/20

Abstract: Embodiments relate to a configurable convolution engine that receives configuration information to perform convolution and other deep machine learning operations on streaming input data of various formats. The convolution engine may include two convolution circuits that each generate a stream of values by applying convolution kernels to input data. The stream of values may each define multiple channels of image data. A channel merge circuit interleaves the streams of values from the convolution circuits to generate an output stream of output values. The output stream includes the data multiple channels of each input stream arranged in an interleaved manner.

公开(公告)号：US20180005344A1

公开(公告)日：2018-01-04

申请号：US15198478

申请日：2016-06-30

Applicant: Apple Inc.

Inventor： Suk Hwan Lim ,  Junji Sugisawa ,  Muge Wang

IPC: G06T1/20 ,  G06F17/15

CPC classification number: G06F17/153

Abstract: Embodiments of the present disclosure relate to a configurable convolution engine that receives configuration information to perform convolution or its variant operations on streaming input data of various formats. To process streaming input data, input data of multiple channels are received and stored in an input buffer circuit in an interleaved manner. Data values of the interleaved input data are retrieved and forwarded to multiplier circuits where multiplication with a corresponding filter element of a kernel is performed. Varying number of kernels with different sizes and sparsity can also be used for the convolution operations.