知搜-全球专利检索

  1. Login
  2. Sign Up

Search Results

12 records (took 0.013 seconds)

    Wide and deep machine learning models
    2.
    发明授权
    Wide and deep machine learning models 有权

    公开(公告)号：US10762422B2

    公开(公告)日：2020-09-01

    申请号：US15394668

    申请日：2016-12-29

    Applicant: Google LLC

    Inventor： Tal Shaked ,  Rohan Anil ,  Hrishikesh Balkrishna Aradhye ,  Mustafa Ispir ,  Glen Anderson ,  Wei Chai ,  Mehmet Levent Koc ,  Jeremiah Harmsen ,  Xiaobing Liu ,  Gregory Sean Corrado ,  Tushar Deepak Chandra ,  Heng-Tze Cheng

    IPC: G06N3/08 ,  G06N3/04

    Abstract: A system includes one or more computers and one or more storage devices storing instructions that when executed by the one or more computers cause the computers to implement a combined machine learning model for processing an input including multiple features to generate a predicted output for the machine learning input. The combined model includes: a deep machine learning model configured to process the features to generate a deep model output; a wide machine learning model configured to process the features to generate a wide model output; and a combining layer configured to process the deep model output generated by the deep machine learning model and the wide model output generated by the wide machine learning model to generate the predicted output, in which the deep model and the wide model have been trained jointly on training data to generate the deep model output and the wide model output.

    Knowledge Distillation Via Learning to Predict Principal Components Coefficients
    3.
    发明申请
    Knowledge Distillation Via Learning to Predict Principal Components Coefficients 有权

    公开(公告)号：US20250005453A1

    公开(公告)日：2025-01-02

    申请号：US18710814

    申请日：2022-12-12

    Applicant: Google LLC

    Inventor： Ehsan Amid ,  Christopher James Fifty ,  Manfred Klaus Warmuth ,  Rohan Anil

    IPC: G06N20/00

    Abstract: Provided is an approach for knowledge distillation based on exporting Principal Components approximations (e.g., Bregman representations) of one or more layer-wise representations of the teacher model. In particular, the present disclosure provides an extension to the original Bregman PCA formulation by incorporating a mean vector and orthonormalizing the principal directions with respect to the geometry of the local convex function around the mean. This extended formulation allows viewing the learned representation as a dense layer, thus casting the problem as learning the linear coefficients of the compressed examples, as the input to this layer, by the student network. Example empirical data indicates that example implementations of the approach improve performance when compared to typical teacher-student training using soft labels.

    Heterogeneous Federated Learning Via Multi-Directional Knowledge Distillation
    9.
    发明公开
    Heterogeneous Federated Learning Via Multi-Directional Knowledge Distillation 审中-公开

    公开(公告)号：US20240249193A1

    公开(公告)日：2024-07-25

    申请号：US18417947

    申请日：2024-01-19

    Applicant: Google LLC

    Inventor： Jared Alexander Lichtarge ,  Rajiv Mathews ,  Rohan Anil ,  Ehsan Amid ,  Shankar Kumar

    IPC: G06N20/00

    CPC classification number: G06N20/00

    Abstract: Generally, the present disclosure is directed to enhanced federated learning (FL) that employs a set of clients with varying amounts of computational resources (e.g., system memory, storage, and processing bandwidth). To overcome limitations of conventional FL methods that employ a set of clients with varying amounts of computational resources, the embodiments run multi-directional knowledge distillation between the server models produced by each federated averaging (FedAvg) pool, using unlabeled server data as the distillation dataset. By co-distilling the two (or more) models frequently over the course of FedAvg rounds, information is shared between the pools without sharing model parameters. This leads to increased performance and faster convergence (in fewer federated rounds).

    DECENTRALIZED LEARNING OF MACHINE LEARNING MODEL(S) THROUGH UTILIZATION OF STALE UPDATES(S) RECEIVED FROM STRAGGLER COMPUTING DEVICE(S)
    10.
    发明公开
    DECENTRALIZED LEARNING OF MACHINE LEARNING MODEL(S) THROUGH UTILIZATION OF STALE UPDATES(S) RECEIVED FROM STRAGGLER COMPUTING DEVICE(S) 审中-公开

    公开(公告)号：US20240095582A1

    公开(公告)日：2024-03-21

    申请号：US18075757

    申请日：2022-12-06

    Applicant: GOOGLE LLC

    Inventor： Andrew Hard ,  Sean Augenstein ,  Rohan Anil ,  Rajiv Mathews ,  Lara McConnaughey ,  Ehsan Amid ,  Antonious Girgis

    IPC: G06N20/00

    CPC classification number: G06N20/00

    Abstract: During a round of decentralized learning for updating of a global machine learning (ML) model, remote processor(s) of a remote system may transmit, to a population of computing devices, primary weights for a primary version of the global ML model, and cause each of the computing devices to generate a corresponding update for the primary version of the global ML model. Further, the remote processor(s) may cause the primary version of the global ML model to be updated based on the corresponding updates that are received during the round of decentralized learning. However, the remote processor(s) may receive other corresponding updates subsequent to the round of decentralized learning. Accordingly, various techniques described herein (e.g., FARe-DUST, FeAST on MSG, and/or other techniques) enable the other corresponding updates to be utilized in achieving a final version of the global ML model.

Patent Agency Ranking