公开(公告)号：US20240331371A1

公开(公告)日：2024-10-03

申请号：US18573973

申请日：2021-11-30

Applicant: Intel Corporation

Inventor： Yurong Chen ,  Anbang Yao ,  Ming Lu ,  Dongqi Cai ,  Xiaolong Liu

IPC: G06V10/82

CPC classification number: G06V10/82

Abstract: Methods and apparatus to perform parallel double-batched self-distillation in resource-constrained image recognition environments are disclosed herein. Example apparatus disclosed herein are to identify a source data batch and an augmented data batch, the augmented data generated based on at least one data augmentation technique. Disclosed example apparatus is also to share one or more parameters between a student neural network corresponding to the source data batch and a teacher neural network corresponding to the augmented data batch, the one or more parameters including one or more convolution layers to be shared between the teacher neural network and the student neural network. Disclosed example apparatus is further to align knowledge corresponding to the teacher neural network and the student neural network, the knowledge corresponding to the one or more parameters shared between the student neural network and the teacher neural network.

公开(公告)号：US20240273873A1

公开(公告)日：2024-08-15

申请号：US18563305

申请日：2021-09-01

Applicant: Intel Corporation

Inventor： Dongqi Cai ,  Anbang Yao ,  Yurong Chen

IPC: G06V10/77 ,  G06V10/32 ,  G06V10/82

CPC classification number: G06V10/7715 ,  G06V10/32 ,  G06V10/82

Abstract: Techniques related to application of deep neural networks to video for video recognition and understanding are discussed. A feature map of a deep neural network for a current time stamp of input video is standardized to a standardized feature map and pooled to a feature vector. The feature vector and transform parameters for a prior time stamp are used to generate transform parameters for the current time stamp based on application of a meta temporal relay. The resultant current time stamp transform parameters, such as a hidden state and a cell state of the meta temporal relay, are used to transform the standardized feature map to a normalized feature map for use by a subsequent layer of the deep neural network.

公开(公告)号：US11790223B2

公开(公告)日：2023-10-17

申请号：US16475076

申请日：2017-04-07

Applicant: INTEL CORPORATION

Inventor： Libin Wang ,  Yiwen Guo ,  Anbang Yao ,  Dongqi Cai ,  Lin Xu ,  Ping Hu ,  Shandong Wang ,  Wenhua Cheng ,  Yurong Chen

IPC: G06N3/08 ,  G06N3/063 ,  G06T1/20 ,  G06F18/21 ,  G06F18/214 ,  G06N3/045

CPC classification number: G06N3/08 ,  G06F18/217 ,  G06F18/2148 ,  G06N3/045 ,  G06N3/063 ,  G06T1/20

Abstract: Methods and systems are disclosed for boosting deep neural networks for deep learning. In one example, in a deep neural network including a first shallow network and a second shallow network, a first training sample is processed by the first shallow network using equal weights. A loss for the first shallow network is determined based on the processed training sample using equal weights. Weights for the second shallow network are adjusted based on the determined loss for the first shallow network. A second training sample is processed by the second shallow network using the adjusted weights. In another example, in a deep neural network including a first weak network and a second weak network, a first subset of training samples is processed by the first weak network using initialized weights. A classification error for the first weak network on the first subset of training samples is determined. The second weak network is boosted using the determined classification error of the first weak network with adjusted weights. A second subset of training samples is processed by the second weak network using the adjusted weights.

公开(公告)号：US20230274132A1

公开(公告)日：2023-08-31

申请号：US18005804

申请日：2020-08-26

Applicant: Intel Corporation

Inventor： Dongqi Cai ,  Anbang Yao ,  Yurong Chen

IPC: G06N3/08

CPC classification number: G06N3/08

Abstract: Methods, apparatus, systems, and articles of manufacture to dynamically normalize data in neural networks are disclosed. An apparatus for use with a machine learning model includes at least one normalization calculator to generate a plurality of alternate normalized outputs associated with input data for the machine learning model. Different ones of the alternate normalized outputs based on different normalization techniques. The apparatus further includes a soft weighting engine to generate a plurality of soft weights based on the input data. The apparatus also includes a normalized output generator to generate a final normalized output based on the plurality of alternate normalized outputs and the plurality of soft weights.

公开(公告)号：US11551335B2

公开(公告)日：2023-01-10

申请号：US16474848

申请日：2017-04-07

Applicant: Intel Corporation

Inventor： Lin Xu ,  Liu Yang ,  Anbang Yao ,  Dongqi Cai ,  Libin Wang ,  Ping Hu ,  Shandong Wang ,  Wenhua Cheng ,  Yiwen Guo ,  Yurong Chen

IPC: G06T5/00 ,  G06N20/10 ,  G06F9/50 ,  G06N3/08 ,  G06N5/04 ,  H04N9/07 ,  H04N9/64

Abstract: Methods and systems are disclosed using camera devices for deep channel and Convolutional Neural Network (CNN) images and formats. In one example, image values are captured by a color sensor array in an image capturing device or camera. The image values provide color channel data. The captured image values by the color sensor array are input to a CNN having at least one CNN layer. The CNN provides CNN channel data for each layer. The color channel data and CNN channel data is to form a deep channel image that stored in a memory. In another example, image values are captured by sensor array. The captured image values by the sensor array are input a CNN having a first CNN layer. An output is generated at the first CNN layer using the captured image values by the color sensor array. The output of the first CNN layer is stored as a feature map of the captured image.

公开(公告)号：US20220207359A1

公开(公告)日：2022-06-30

申请号：US17485406

申请日：2021-09-25

Applicant: Intel Corporation

Inventor： Anbang Yao ,  Dongqi Cai ,  Yurong Chen ,  Wenjian Shao ,  Feng Chen

IPC: G06N3/08 ,  G06T3/40 ,  G06T3/00 ,  G06N3/04

Abstract: Embodiments are generally directed to methods and apparatuses for dynamic normalization and relay in a neural network. An embodiment of an apparatus for dynamic normalization and relay in a neural network including a hyper normalization layer comprises: a compute engine to: generate a hidden state and a cell state for the hyper normalization layer based on an input feature map for the hyper normalization layer as well as a previous hidden state and a previous cell state; and normalize the input feature map in the hyper normalization layer with the hidden state and the cell state for the hyper normalization layer.

公开(公告)号：US11341368B2

公开(公告)日：2022-05-24

申请号：US16475079

申请日：2017-04-07

Applicant: INTEL CORPORATION

Inventor： Anbang Yao ,  Shandong Wang ,  Wenhua Cheng ,  Dongqi Cai ,  Libin Wang ,  Lin Xu ,  Ping Hu ,  Yiwen Guo ,  Liu Yang ,  Yuqing Hou ,  Zhou Su ,  Yurong Chen

IPC: G06K9/62 ,  G06N3/08 ,  G06T1/20 ,  G06V10/94

Abstract: Methods and systems for advanced and augmented training of deep neural networks (DNNs) using synthetic data and innovative generative networks. A method includes training a DNN using synthetic data, training a plurality of DNNs using context data, associating features of the DNNs trained using context data with features of the DNN trained with synthetic data, and generating an augmented DNN using the associated features.

公开(公告)号：US11107189B2

公开(公告)日：2021-08-31

申请号：US16474927

申请日：2017-04-07

Applicant: INTEL CORPORATION

Inventor： Shandong Wang ,  Yiwen Guo ,  Anbang Yao ,  Dongqi Cai ,  Libin Wang ,  Lin Xu ,  Ping Hu ,  Wenhua Cheng ,  Yurong Chen

IPC: G06K9/00 ,  G06T3/40 ,  G06N20/20 ,  G06N20/10 ,  G06K9/62 ,  G06N3/04 ,  G06N3/08 ,  G06N5/04 ,  G06T1/20

Abstract: Methods and systems are disclosed using improved Convolutional Neural Networks (CNN) for image processing. In one example, an input image is down-sampled into smaller images with a smaller resolution than the input image. The down-sampled smaller images are processed by a CNN having a last layer with a reduced number of nodes than a last layer of a full CNN used to process the input image at a full resolution. A result is outputted based on the processed down-sampled smaller images by the CNN having a last layer with a reduced number of nodes. In another example, shallow CNN networks are built randomly. The randomly built shallow CNN networks are combined to imitate a trained deep neural network (DNN).

公开(公告)号：US20200279156A1

公开(公告)日：2020-09-03

申请号：US16645425

申请日：2017-10-09

Applicant: INTEL CORPORATION

Inventor： Dongqi Cai ,  Anbang Yao ,  Ping Hu ,  Shandong Wang ,  Yurong Chen

IPC: G06N3/08 ,  G06N3/04

Abstract: A system to perform multi-modal analysis has at least three distinct characteristics: an early abstraction layer for each data modality integrating homogeneous feature cues coming from different deep learning architectures for that data modality, a late abstraction layer for further integrating heterogeneous features extracted from different models or data modalities and output from the early abstraction layer, and a propagation-down strategy for joint network training in an end-to-end manner. The system is thus able to consider correlations among homogeneous features and correlations among heterogenous features at different levels of abstraction. The system further extracts and fuses discriminative information contained in these models and modalities for high performance emotion recognition.

公开(公告)号：US20200026988A1

公开(公告)日：2020-01-23

申请号：US16475075

申请日：2017-04-07

Applicant: INTEL CORPORATION

Inventor： Yiwen Guo ,  Anbang Yao ,  Dongqi Cai ,  Libin Wang ,  Lin Xu ,  Ping Hu ,  Shangong Wang ,  Wenhua Cheng ,  Wenhua Cheng ,  Yurong Chen

IPC: G06N3/04 ,  G06K9/62 ,  G06F17/18 ,  G06T1/20

Abstract: Methods and systems are disclosed using improved training and learning for deep neural networks. In one example, a deep neural network includes a plurality of layers, and each layer has a plurality of nodes. For each L layer in the plurality of layers, the nodes of each L layer are randomly connected to nodes in a L+1 layer. For each L+1 layer in the plurality of layers, the nodes of each L+1 layer are connected to nodes in a subsequent L layer in a one-to-one manner. Parameters related to the nodes of each L layer are fixed. Parameters related to the nodes of each L+1 layers are updated, and L is an integer starting with 1. In another example, a deep neural network includes an input layer, output layer, and a plurality of hidden layers. Inputs for the input layer and labels for the output layer are determined related to a first sample. Similarity between different pairs of inputs and labels between a second sample with the first sample is estimated using Gaussian regression process.