公开(公告)号：US20230205994A1

公开(公告)日：2023-06-29

申请号：US17561581

申请日：2021-12-23

Applicant: Google LLC

Inventor： Jason Weng Wei ,  Maarten Paul Bosma ,  Yuzhe Zhao, JR. ,  Kelvin Gu ,  Quoc V. Le

IPC: G06F40/284 ,  G06F40/30 ,  G06N3/10 ,  G06N5/04

CPC classification number: G06F40/284 ,  G06F40/30 ,  G06N3/10 ,  G06N5/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing a machine learning task on an input to generate an output. In one aspect, one of the method includes receiving input data that describes an input of a machine learning task; receiving candidate output data that describes a set of candidate classification outputs of the machine learning task for the input; generating an input sequence that includes the input and the set of candidate classification outputs; processing the input sequence using a neural network to generate a network output that specifies a respective score for each candidate classification output in the set of candidate classification outputs; and generating an output of the machine learning task for the input, comprising selecting, as the output, a selected candidate classification output from the set of candidate classification outputs using the respective scores.

公开(公告)号：US20240256965A1

公开(公告)日：2024-08-01

申请号：US18424624

申请日：2024-01-26

Applicant: Google LLC

Inventor： Hyung Won Chung ,  Barret Zoph ,  Dengyong Zhou ,  Liam Fedus ,  Shayne Longpre ,  Le Hou ,  Yi Tay ,  Jason Weng Wei ,  Siddhartha Brahma ,  Quoc V. Le

IPC: G06N20/00

CPC classification number: G06N20/00

Abstract: An example method for training a machine-learned sequence processing model includes obtaining a plurality of training examples for training the machine-learned sequence processing model. For each respective training example of the plurality of training examples, the example method includes: obtaining a respective query associated with the respective training example; inputting the respective query to the machine-learned sequence processing model; obtaining, from the machine-learned sequence processing model a response to the respective query and a trace of intermediate states from the respective query to the response; evaluating the response using a ground truth response associated with the respective training example; evaluating the trace using a ground truth trace associated with the respective training example; and updating one or more parameters of the machine-learned sequence processing model based on the evaluation of the response and based on the evaluation of the trace.

公开(公告)号：US20230394328A1

公开(公告)日：2023-12-07

申请号：US17881746

申请日：2022-08-05

Applicant: Google LLC

Inventor： Jason Weng Wei ,  Dengyong Zhou ,  Dale Eric Schuurmans ,  Quoc V. Le ,  Maarten Paul Bosma ,  Ed Huai-Hsin Chi ,  Olivier Jean Andrè Bousquet ,  Le Hou ,  Nathan Kemp Sekiguchi Scales ,  David J. Bieber ,  Charles Aloysius Sutton ,  Nathanael Martin Schärli ,  Augustus Quadrozzi Odena ,  Sharan Ajit Narang ,  Guy Gur-Ari Krakover ,  Aakanksha Chowdhery ,  Aitor Lewkowycz ,  Jiageng Luan ,  David Martin Dohan ,  Henryk Michalewski ,  Jacob Austin ,  Anders Johan Andreassen ,  Maxwell Isaac Nye ,  Xuezhi Wang

IPC: G06N5/02

CPC classification number: G06N5/022

Abstract: Example embodiments of aspects of the present disclosure provide an example computer-implemented method for improved prompting of a machine-learned model. The example method can include obtaining an instructive sequence descriptive of an instructive query, an instructive response, and an instructive trace of intermediate states from the instructive query to the instructive response. The example method can include inputting, to a machine-learned model, the instructive sequence and an operative query, wherein the machine-learned model is configured to process the operative query with attention over the instructive sequence. The example method can include generating, using the machine-learned model and responsive to the operative query, an operative response.

公开(公告)号：US20230244938A1

公开(公告)日：2023-08-03

申请号：US18160776

申请日：2023-01-27

Applicant: Google LLC

Inventor： Jason Weng Wei ,  Dengyong Zhou ,  Xuezhi Wang ,  Dale Eric Schuurmans ,  Quoc V. Le ,  Maarten Paul Bosma ,  Ed Huai-Hsin Chi ,  Olivier Jean Andrè Bousquet ,  Le Hou ,  Charles Aloysius Sutton ,  Nathanael Martin Schärli ,  Nathan Kemp Sekiguchi Scales ,  Augustus Quadrozzi Odena ,  Sharan Ajit Narang ,  Guy Gur-Ari Krakover ,  Aakanksha Chowdhery ,  David Martin Dohan ,  Aitor Lewkowycz ,  Henryk Michalewski ,  Jiageng Luan ,  David J. Bieber ,  Jacob Austin ,  Anders Johan Andreassen ,  Maxwell Isaac Nye ,  Yi Tay ,  Mostafa Dehghani

IPC: G06N3/08

CPC classification number: G06N3/08

Abstract: An example method for pretraining a machine-learned model is provided. The example method includes obtaining a plurality of different combinations of configuration parameters of a pretraining objective framework. The example method includes generating, using the pretraining objective framework, a plurality of corrupted training examples from one or more training examples, wherein the plurality of corrupted training examples are respectively generated according to the plurality of different combinations. The example method includes inputting the plurality of corrupted training examples into the machine-learned model, wherein the machine-learned model is configured to generate uncorrupted subportions corresponding to corrupted subportions of the corrupted training examples. The example method includes obtaining, from the machine-learned model, a plurality of outputs respectively generated by the machine-learned model based on the plurality of corrupted training examples. The example method includes updating one or more parameters of the machine-learned model based on an evaluation of the plurality of outputs.