公开(公告)号：US11520629B2

公开(公告)日：2022-12-06

申请号：US16776338

申请日：2020-01-29

Applicant: Apple Inc.

Inventor： Francesco Rossi ,  Gaurav Kapoor ,  Michael R. Siracusa ,  William B. March

IPC: G06F9/46 ,  G06F9/50 ,  G06N3/063 ,  G06F8/41 ,  G06F9/48 ,  G06N3/02

Abstract: The subject technology provides for dynamic task allocation for neural network models. The subject technology determines an operation performed at a node of a neural network model. The subject technology assigns an annotation to indicate whether the operation is better performed on a CPU or a GPU based at least in part on hardware capabilities of a target platform. The subject technology determines whether the neural network model includes a second layer. The subject technology, in response to determining that the neural network model includes a second layer, for each node of the second layer of the neural network model, determines a second operation performed at the node. Further the subject technology assigns a second annotation to indicate whether the second operation is better performed on the CPU or the GPU based at least in part on the hardware capabilities of the target platform.

公开(公告)号：US20180349189A1

公开(公告)日：2018-12-06

申请号：US15721716

申请日：2017-09-29

Applicant: Apple Inc.

Inventor： Francesco Rossi ,  Gaurav Kapoor ,  Michael R. Siracusa ,  William B. March

IPC: G06F9/50 ,  G06N3/02

Abstract: The subject technology provides for dynamic task allocation for neural network models. The subject technology determines an operation performed at a node of a neural network model. The subject technology assigns an annotation to indicate whether the operation is better performed on a CPU or a GPU based at least in part on hardware capabilities of a target platform. The subject technology determines whether the neural network model includes a second layer. The subject technology, in response to determining that the neural network model includes a second layer, for each node of the second layer of the neural network model, determines a second operation performed at the node. Further the subject technology assigns a second annotation to indicate whether the second operation is better performed on the CPU or the GPU based at least in part on the hardware capabilities of the target platform.

公开(公告)号：US10664963B1

公开(公告)日：2020-05-26

申请号：US16032879

申请日：2018-07-11

Applicant: Apple Inc.

Inventor： Francesco Rossi ,  Xiaohuan C. Wang ,  Bartlomiej W. Rymkowski ,  Xiaojin Shi ,  Marco Zuliani ,  Alexey Marinichev

IPC: G06T5/50 ,  G06N3/08 ,  G06N3/04

Abstract: Artistic styles extracted from one or more source images may be applied to one or more target images, e.g., in the form of stylized images and/or stylized video sequences. The extracted artistic style may be stored as a plurality of layers in a neural network, which neural network may be further optimized, e.g., via the fusion of various elements of the network's architectures. An optimized network architecture may be determined for each processing environment in which the network will be applied. The artistic style may be applied to the obtained images and/or video sequence of images using various optimization methods, such as the use of scalars to control the resolution of the unstylized and stylized images, temporal consistency constraints, as well as the use of dynamically adjustable or selectable versions of Deep Neural Networks (DNN) that are responsive to system performance parameters, such as available processing resources and thermal capacity.

公开(公告)号：US20180088996A1

公开(公告)日：2018-03-29

申请号：US15711781

申请日：2017-09-21

Applicant: Apple Inc.

Inventor： Francesco Rossi ,  Marco Zuliani

IPC: G06F9/50

CPC classification number: G06F9/5016

Abstract: A method may include accessing a data processing architecture associated with a neural network to determine dependencies between intermediate data layers of the neural network; obtaining dimensions of the intermediate data layers in the neural network; calculating a minimum number of data storage portions for executing the neural network based on the dependencies; determining a memory allocation size for each respective data storage portion of the data storage portions based on the dimensions and dependencies; allocating memory on a storage device for each data storage portion in accordance with its respective determined memory allocation size.

公开(公告)号：US12175375B2

公开(公告)日：2024-12-24

申请号：US17903991

申请日：2022-09-06

Applicant: Apple Inc.

Inventor： Gaurav Kapoor ,  Cecile M. Foret ,  Francesco Rossi ,  Kit-Man Wan ,  Umesh S. Vaishampayan ,  Etienne Belanger ,  Albert Antony ,  Alexey Marinichev ,  Marco Zuliani ,  Xiaojin Shi

IPC: G06N3/10 ,  G06F8/41 ,  G06F9/50 ,  G06N3/04 ,  G06N3/063 ,  G06N3/08

Abstract: The subject technology provides receiving a neural network (NN) model to be executed on a target platform, the NN model including multiple layers that include operations and some of the operations being executable on multiple processors of the target platform. The subject technology further sorts the operations from the multiple layers in a particular order based at least in part on grouping the operations that are executable by a particular processor of the multiple processors. The subject technology determines, based at least in part on a cost of transferring the operations between the multiple processors, an assignment of one of the multiple processors for each of the sorted operations of each of the layers in a manner that minimizes a total cost of executing the operations. Further, for each layer of the NN model, the subject technology includes an annotation to indicate the processor assigned for each of the operations.

公开(公告)号：US12051006B2

公开(公告)日：2024-07-30

申请号：US17903991

申请日：2022-09-06

Applicant: Apple Inc.

Inventor： Gaurav Kapoor ,  Cecile M. Foret ,  Francesco Rossi ,  Kit-Man Wan ,  Umesh S. Vaishampayan ,  Etienne Belanger ,  Albert Antony ,  Alexey Marinichev ,  Marco Zuliani ,  Xiaojin Shi

IPC: G06N3/10 ,  G06F8/41 ,  G06F9/50 ,  G06N3/04 ,  G06N3/063 ,  G06N3/08

CPC classification number: G06N3/10 ,  G06F8/41 ,  G06F8/443 ,  G06F8/4441 ,  G06N3/04 ,  G06N3/063 ,  G06N3/08 ,  G06F9/50 ,  G06N3/08 ,  G06N3/063 ,  G06N3/04 ,  G06N3/10

Abstract: The subject technology provides receiving a neural network (NN) model to be executed on a target platform, the NN model including multiple layers that include operations and some of the operations being executable on multiple processors of the target platform. The subject technology further sorts the operations from the multiple layers in a particular order based at least in part on grouping the operations that are executable by a particular processor of the multiple processors. The subject technology determines, based at least in part on a cost of transferring the operations between the multiple processors, an assignment of one of the multiple processors for each of the sorted operations of each of the layers in a manner that minimizes a total cost of executing the operations. Further, for each layer of the NN model, the subject technology includes an annotation to indicate the processor assigned for each of the operations.

公开(公告)号：US11907760B2

公开(公告)日：2024-02-20

申请号：US15711781

申请日：2017-09-21

Applicant: Apple Inc.

Inventor： Francesco Rossi ,  Marco Zuliani

IPC: G06F9/50

CPC classification number: G06F9/5016

Abstract: A method may include accessing a data processing architecture associated with a neural network to determine dependencies between intermediate data layers of the neural network; obtaining dimensions of the intermediate data layers in the neural network; calculating a minimum number of data storage portions for executing the neural network based on the dependencies; determining a memory allocation size for each respective data storage portion of the data storage portions based on the dimensions and dependencies; allocating memory on a storage device for each data storage portion in accordance with its respective determined memory allocation size.

公开(公告)号：US11367163B2

公开(公告)日：2022-06-21

申请号：US16794824

申请日：2020-02-19

Applicant: Apple Inc.

Inventor： Francesco Rossi ,  Marco Zuliani ,  Bartlomiej W. Rymkowski ,  Albert Antony ,  Brian P. Keene ,  Xiaojin Shi

IPC: G06T3/40 ,  G06T11/60 ,  G06N3/04

Abstract: Artistic styles extracted from source images may be applied to target images to generate stylized images and/or video sequences. The extracted artistic styles may be stored as a plurality of layers in one or more neural networks, which neural networks may be further optimized, e.g., via the fusion of various elements of the networks' architectures. The artistic style may be applied to the target images and/or video sequences using various optimization methods, such as the use of a first version of the neural network by a first processing device at a first resolution to generate one or more sets of parameters (e.g., scaling and/or biasing parameters), which parameters may then be mapped for use by a second version of the neural network by a second processing device at a second resolution. Analogous multi-processing device and/or multi-network solutions may also be applied to other complex image processing tasks for increased efficiency.

公开(公告)号：US20200380639A1

公开(公告)日：2020-12-03

申请号：US16794824

申请日：2020-02-19

Applicant: Apple Inc.

Inventor： Francesco Rossi ,  Marco Zuliani ,  Bartlomiej W. Rymkowski ,  Albert Antony ,  Brian P. Keene ,  Xiaojin Shi

IPC: G06T3/40 ,  G06T11/60 ,  G06N3/04

Abstract: Artistic styles extracted from source images may be applied to target images to generate stylized images and/or video sequences. The extracted artistic styles may be stored as a plurality of layers in one or more neural networks, which neural networks may be further optimized, e.g., via the fusion of various elements of the networks' architectures. The artistic style may be applied to the target images and/or video sequences using various optimization methods, such as the use of a first version of the neural network by a first processing device at a first resolution to generate one or more sets of parameters (e.g., scaling and/or biasing parameters), which parameters may then be mapped for use by a second version of the neural network by a second processing device at a second resolution. Analogous multi-processing device and/or multi-network solutions may also be applied to other complex image processing tasks for increased efficiency.

公开(公告)号：US20200372408A1

公开(公告)日：2020-11-26

申请号：US16878254

申请日：2020-05-19

Applicant: Apple Inc.

Inventor： Francesco Rossi ,  Vignesh Jagadeesh ,  Vinay Sharma ,  Marco Zuliani ,  Xiaojin Shi ,  Benjamin Poulain

IPC: G06N20/00 ,  G06F9/38

Abstract: A method includes receiving input data at a trained machine learning model that includes a common part and task-specific parts, receiving an execution instruction that identifies one or more processing tasks to be performed, processing the input data using the common part of the trained machine learning model to generate intermediate data, and processing the intermediate data using one or more of the task-specific parts of the trained machine learning model based on the execution instruction to generate one or more outputs.