MODEL DECORRELATION AND SUBSPACING FOR FEDERATED LEARNING

    公开(公告)号:US20240104367A1

    公开(公告)日:2024-03-28

    申请号:US17934098

    申请日:2022-09-21

    CPC classification number: G06N3/08 H04B17/3913

    Abstract: Certain aspects of the present disclosure provide techniques and apparatus for training a machine learning model. An example method generally includes partitioning a machine learning model into a plurality of partitions. A request to update a respective partition of the plurality of partitions in the machine learning model is transmitted to each respective participating device of a plurality of participating devices in a federated learning scheme, and the request may specify that the respective partition is to be updated based on unique data at the respective participating device. Updates to one or more partitions in the machine learning model are received from the plurality of participating devices, and the machine learning model is updated based on the received updates.

    FEDERATED LEARNING SURROGATION WITH TRUSTED SERVER

    公开(公告)号:US20240095513A1

    公开(公告)日:2024-03-21

    申请号:US17932809

    申请日:2022-09-16

    CPC classification number: G06N3/08

    Abstract: Certain aspects of the present disclosure provide techniques and apparatus for surrogated federated learning. A set of intermediate activations is received at a trusted server from a node device, where the node device generated the set of intermediate activations using a first set of layers of a neural network. One or more weights associated with a second set of layers of the neural network are refined using the set of intermediate activations, and one or more weight updates corresponding to the refined one or more weights are transmitted to a federated learning system.

    DESPARSIFIED CONVOLUTION FOR SPARSE ACTIVATIONS

    公开(公告)号:US20240046078A1

    公开(公告)日:2024-02-08

    申请号:US17817552

    申请日:2022-08-04

    CPC classification number: G06N3/0481

    Abstract: Certain aspects of the present disclosure provide techniques for desparsified convolution. An activation tensor is received, and a convolution output is generated for the activation tensor, comprising: selecting a subset of weight elements, corresponding to a set of non-zero elements in the activation tensor, from a weight tensor, and multiplying the set of non-zero elements and the set of weight elements.

    EFFICIENT NEURAL-NETWORK-BASED PROCESSING OF VISUAL CONTENT

    公开(公告)号:US20230215157A1

    公开(公告)日:2023-07-06

    申请号:US18150578

    申请日:2023-01-05

    CPC classification number: G06V10/82 G06V10/77 G06V10/75

    Abstract: Certain aspects of the present disclosure provide techniques and apparatus for efficient processing of visual content using machine learning models. An example method generally includes generating, from an input, an embedding tensor for the input. The embedding tensor for the input is projected into a reduced-dimensional space projection of the embedding tensor based on a projection matrix. An attention value for the input is derived based on the reduced-dimensional space projection of the embedding tensor and a non-linear attention function. A match, in the reduced-dimensional space, is identified between a portion of the input and a corresponding portion of a target against which the input is evaluated based on the attention value for the input. One or more actions are taken based on identifying the match.

    CONVOLUTION WITH KERNEL EXPANSION AND TENSOR ACCUMULATION

    公开(公告)号:US20220391702A1

    公开(公告)日:2022-12-08

    申请号:US17805021

    申请日:2022-06-01

    Abstract: Certain aspects of the present disclosure provide techniques for kernel expansion. An input data tensor is received at a first layer in a neural network, and a first convolution is performed for a first kernel, where the first kernel has a size greater than a preferred size. Performing the first convolution comprises generating a plurality of intermediate tensors by performing a plurality of intermediate convolutions using a plurality of intermediate kernels with a size of the preferred size, and accumulating the plurality of intermediate tensors to generate an output tensor for the first convolution.

    ENERGY EFFICIENT MACHINE LEARNING MODELS

    公开(公告)号:US20210158145A1

    公开(公告)日:2021-05-27

    申请号:US16694442

    申请日:2019-11-25

    Abstract: Aspects described herein provide a method including: receiving input data at a machine learning model, comprising: a plurality of processing layers; a plurality of gate logics; a plurality of gates; and a fully connected layer; determining based on a plurality of gate parameters associated with the plurality of gate logics, a subset of the plurality of processing layers with which to process the input data; processing the input data with the subset of the plurality of processing layers and the fully connected layer to generate an inference; determining a prediction loss based on the inference and a training label associated with the input data; determining an energy loss based on the subset of the plurality of processing layers used to process the input data; and optimizing the machine learning model based on: the prediction loss; the energy loss; and a prior probability associated with the training label.

    EFFICIENT INFERENCING WITH PIECEWISE POINTWISE CONVOLUTION

    公开(公告)号:US20210019593A1

    公开(公告)日:2021-01-21

    申请号:US16932496

    申请日:2020-07-17

    Abstract: Certain aspects of the present disclosure provide techniques for performing piecewise pointwise convolution, comprising: performing a first piecewise pointwise convolution on a first subset of data received via a first branch input at a piecewise pointwise convolution layer of a convolutional neural network (CNN) model; performing a second piecewise pointwise convolution on a second subset of data received via a second branch input at the piecewise pointwise convolution layer; determining a piecewise pointwise convolution output by summing a result of the first piecewise pointwise convolution and a result of the second piecewise pointwise convolution; and providing the piecewise pointwise convolution output to a second layer of the CNN model.

Patent Agency Ranking