公开(公告)号：US10747843B2

公开(公告)日：2020-08-18

申请号：US16791926

申请日：2020-02-14

Applicant: Apple Inc.

Inventor： Suk Hwan Lim ,  Junji Sugisawa ,  Muge Wang

IPC: G06F17/15 ,  G06T1/20 ,  G06T5/00 ,  G06T5/20

Abstract: Embodiments of the present disclosure relate to a configurable convolution engine that receives configuration information to perform convolution or its variant operations on streaming input data of various formats. To process streaming input data, input data of multiple channels are received and stored in an input buffer circuit in an interleaved manner. Data values of the interleaved input data are retrieved and forwarded to multiplier circuits where multiplication with a corresponding filter element of a kernel is performed. Varying number of kernels with different sizes and sparsity can also be used for the convolution operations.

公开(公告)号：US20200184000A1

公开(公告)日：2020-06-11

申请号：US16791926

申请日：2020-02-14

Applicant: Apple Inc.

Inventor： Suk Hwan Lim ,  Junji Sugisawa ,  Muge Wang

IPC: G06F17/15 ,  G06T5/20 ,  G06T5/00 ,  G06T1/20

Abstract: Embodiments of the present disclosure relate to a configurable convolution engine that receives configuration information to perform convolution or its variant operations on streaming input data of various formats. To process streaming input data, input data of multiple channels are received and stored in an input buffer circuit in an interleaved manner. Data values of the interleaved input data are retrieved and forwarded to multiplier circuits where multiplication with a corresponding filter element of a kernel is performed. Varying number of kernels with different sizes and sparsity can also be used for the convolution operations.

公开(公告)号：US20200167889A1

公开(公告)日：2020-05-28

申请号：US16664120

申请日：2019-10-25

Applicant: Apple Inc.

Inventor： Sung Hee Park ,  Muge Wang ,  Junji Sugisawa

IPC: G06T1/20 ,  G06K9/46 ,  G06K9/00 ,  G06N3/063 ,  G06F17/15 ,  G06T5/20 ,  G06T5/00 ,  G06N3/08 ,  G06N3/04

Abstract: Embodiments relate to a configurable convolution engine that receives configuration information to perform convolution and other deep machine learning operations on streaming input data of various formats. The convolution engine may include two convolution circuits that each generate a stream of values by applying convolution kernels to input data. The stream of values may each define one or more channels of image data. A channel merge circuit combines the streams of values from each convolution circuit in accordance with a selected mode of operation. In one mode, the first and second streams from the convolution circuits are merged into an output stream having the combined channels of the first and second streams in an interleaved manner. In another mode, the first stream from the first convolution circuit is fed into the input of the second convolution circuit.

公开(公告)号：US10606918B2

公开(公告)日：2020-03-31

申请号：US16664096

申请日：2019-10-25

Applicant: Apple Inc.

Inventor： Suk Hwan Lim ,  Junji Sugisawa ,  Muge Wang

IPC: G06F17/15 ,  G06T1/20 ,  G06T5/00 ,  G06T5/20

Abstract: Embodiments of the present disclosure relate to a configurable convolution engine that receives configuration information to perform convolution or its variant operations on streaming input data of various formats. To process streaming input data, input data of multiple channels are received and stored in an input buffer circuit in an interleaved manner. Data values of the interleaved input data are retrieved and forwarded to multiplier circuits where multiplication with a corresponding filter element of a kernel is performed. Varying number of kernels with different sizes and sparsity can also be used for the convolution operations.

公开(公告)号：US10685421B1

公开(公告)日：2020-06-16

申请号：US16664120

申请日：2019-10-25

Applicant: Apple Inc.

Inventor： Sung Hee Park ,  Muge Wang ,  Junji Sugisawa

IPC: G06F17/15 ,  G06T1/20 ,  G06K9/46 ,  G06K9/00 ,  G06N3/04 ,  G06T5/20 ,  G06T5/00 ,  G06N3/08 ,  G06N3/063

Abstract: Embodiments relate to a configurable convolution engine that receives configuration information to perform convolution and other deep machine learning operations on streaming input data of various formats. The convolution engine may include two convolution circuits that each generate a stream of values by applying convolution kernels to input data. The stream of values may each define one or more channels of image data. A channel merge circuit combines the streams of values from each convolution circuit in accordance with a selected mode of operation. In one mode, the first and second streams from the convolution circuits are merged into an output stream having the combined channels of the first and second streams in an interleaved manner. In another mode, the first stream from the first convolution circuit is fed into the input of the second convolution circuit.

公开(公告)号：US20200057789A1

公开(公告)日：2020-02-20

申请号：US16664096

申请日：2019-10-25

Applicant: Apple Inc.

Inventor： Suk Hwan Lim ,  Junji Sugisawa ,  Muge Wang

IPC: G06F17/15 ,  G06T5/20 ,  G06T5/00

Abstract: Embodiments of the present disclosure relate to a configurable convolution engine that receives configuration information to perform convolution or its variant operations on streaming input data of various formats. To process streaming input data, input data of multiple channels are received and stored in an input buffer circuit in an interleaved manner. Data values of the interleaved input data are retrieved and forwarded to multiplier circuits where multiplication with a corresponding filter element of a kernel is performed. Varying number of kernels with different sizes and sparsity can also be used for the convolution operations.

公开(公告)号：US20200051256A1

公开(公告)日：2020-02-13

申请号：US16100780

申请日：2018-08-10

Applicant: Apple Inc.

Inventor： Muge Wang ,  Junji Sugisawa

IPC: G06T7/32 ,  G06T7/00 ,  G06T7/20 ,  G06T5/00 ,  G06K9/62 ,  G06T7/33

Abstract: Embodiments relate to a normalized cross correlation (NCC) circuit that can perform a normalized cross correlation between input patch data and kernel data. An interface circuit of an image signal processor receives input patch data from a source. Input patch data is data that represents a portion of a frame of image data from the source. The NCC circuit includes a filtering circuit and a normalization circuit. The filtering circuit receives the input patch data from the interface circuit and performs a convolution on the received input patch data or processed patch data derived from the input patch data with kernel data to produce convolution output data. The normalization circuit computes a normalized score output based on the convolution output data and the kernel data. The normalized score output includes normalization scores for each location of the convolution output data.

公开(公告)号：US20180005344A1

公开(公告)日：2018-01-04

申请号：US15198478

申请日：2016-06-30

Applicant: Apple Inc.

Inventor： Suk Hwan Lim ,  Junji Sugisawa ,  Muge Wang

IPC: G06T1/20 ,  G06F17/15

CPC classification number: G06F17/153

Abstract: Embodiments of the present disclosure relate to a configurable convolution engine that receives configuration information to perform convolution or its variant operations on streaming input data of various formats. To process streaming input data, input data of multiple channels are received and stored in an input buffer circuit in an interleaved manner. Data values of the interleaved input data are retrieved and forwarded to multiplier circuits where multiplication with a corresponding filter element of a kernel is performed. Varying number of kernels with different sizes and sparsity can also be used for the convolution operations.

公开(公告)号：US10997736B2

公开(公告)日：2021-05-04

申请号：US16100780

申请日：2018-08-10

Applicant: Apple Inc.

Inventor： Muge Wang ,  Junji Sugisawa

IPC: G06T7/32 ,  G06T7/00 ,  G06T7/33 ,  G06T5/00 ,  G06T7/20 ,  G06K9/62

Abstract: Embodiments relate to a normalized cross correlation (NCC) circuit that can perform a normalized cross correlation between input patch data and kernel data. An interface circuit of an image signal processor receives input patch data from a source. Input patch data is data that represents a portion of a frame of image data from the source. The NCC circuit includes a filtering circuit and a normalization circuit. The filtering circuit receives the input patch data from the interface circuit and performs a convolution on the received input patch data or processed patch data derived from the input patch data with kernel data to produce convolution output data. The normalization circuit computes a normalized score output based on the convolution output data and the kernel data. The normalized score output includes normalization scores for each location of the convolution output data.

公开(公告)号：US20190096026A1

公开(公告)日：2019-03-28

申请号：US16203550

申请日：2018-11-28

Applicant: Apple Inc.

Inventor： Sung Hee Park ,  Muge Wang ,  Junji Sugisawa

IPC: G06T1/20 ,  G06K9/46 ,  G06N3/063 ,  G06K9/00 ,  G06T5/00 ,  G06T5/20 ,  G06F17/15

Abstract: Embodiments relate to a configurable convolution engine that receives configuration information to perform convolution and other deep machine learning operations on streaming input data of various formats. The convolution engine may include two convolution circuits that each generate a stream of values by applying convolution kernels to input data. The stream of values may each define one or more channels of image data. A channel merge circuit combines the streams of values from each convolution circuit in accordance with a selected mode of operation. In one mode, the first and second streams from the convolution circuits are merged into an output stream having the combined channels of the first and second streams in an interleaved manner. In another mode, the first stream from the first convolution circuit is fed into the input of the second convolution circuit.