公开(公告)号：US11907674B1

公开(公告)日：2024-02-20

申请号：US18370683

申请日：2023-09-20

Applicant: GOOGLE LLC

Inventor： Oscar Akerlund ,  Evgeny Sluzhaev ,  Golnaz Ghiasi ,  Thang Luong ,  Yifeng Lu ,  Igor Petrovski ,  Ágoston Weisz ,  Wei Yu ,  Rakesh Shivanna ,  Michael Andrew Goodman ,  Apoorv Kulshreshtha ,  Yu Du ,  Amin Ghafouri ,  Sanil Jain ,  Dustin Tran ,  Vikas Peswani ,  YaGuang Li

IPC: G06F40/00 ,  G06F40/40

CPC classification number: G06F40/40

Abstract: Implementations relate to generating multi-modal response(s) through utilization of large language model(s) (LLM(s)). Processor(s) of a system can: receive natural language (NL) based input, generate a multi-modal response that is responsive to the NL based output, and cause the multi-modal response to be rendered. In some implementations, and in generating the multi-modal response, the processor(s) can process, using a LLM, LLM input (e.g., that includes at least the NL based input) to generate LLM output, and determine, based on the LLM output, textual content for inclusion in the multi-modal response and multimedia content for inclusion in the multi-modal response. In some implementations, the multimedia content can be obtained based on a multimedia content tag that is included in the LLM output and that is indicative of the multimedia content. In various implementations, the multimedia content can be interleaved between segments of the textual content.

公开(公告)号：US20240112027A1

公开(公告)日：2024-04-04

申请号：US18477546

申请日：2023-09-28

Applicant: Google LLC

Inventor： Yanqi Zhou ,  Yanping Huang ,  Yifeng Lu ,  Andrew M. Dai ,  Siamak Shakeri ,  Zhifeng Chen ,  James Laudon ,  Quoc V. Le ,  Da Huang ,  Nan Du ,  David Richard So ,  Daiyi Peng ,  Yingwei Cui ,  Jeffrey Adgate Dean ,  Chang Lan

IPC: G06N3/08

CPC classification number: G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing neural architecture search for machine learning models. In one aspect, a method comprises receiving training data for a machine learning, generating a plurality of candidate neural networks for performing the machine learning task, wherein each candidate neural network comprises a plurality of instances of a layer block composed of a plurality of layers, for each candidate neural network, selecting a respective type for each of the plurality of layers from a set of layer types that comprises, training the candidate neural network and evaluating performance scores for the trained candidate neural networks as applied to the machine learning task, and determining a final neural network for performing the machine learning task based at least on the performance scores for the candidate neural networks.

公开(公告)号：US20220180207A1

公开(公告)日：2022-06-09

申请号：US17211200

申请日：2021-03-24

Applicant: Google LLC

Inventor： Chen Liang ,  Da Huang ,  Yifeng Lu

IPC: G06N3/08 ,  G06N3/04

Abstract: Provided is an end-to-end pipeline (e.g., which may be implemented in TensorFlow) which leverages a specialized search space to generate custom models which provide improved time series prediction.

公开(公告)号：US20250139379A1

公开(公告)日：2025-05-01

申请号：US18385270

申请日：2023-10-30

Applicant: GOOGLE LLC

Inventor： Sanil Jain ,  Wei Yu ,  Alessandro Agostini ,  Agoston Weisz ,  Michael Andrew Goodman ,  Attila Dankovics ,  Elle Chae ,  Evgeny Sluzhaev ,  Amin Ghafouri ,  Golnaz Ghiasi ,  Igor Petrovski ,  Konstantin Shagin ,  Marcelo Menegali ,  Oscar Akerlund ,  Rakesh Shivanna ,  Thang Luong ,  Tiffany Chen ,  Vikas Peswani ,  Yifeng Lu

IPC: G06F40/40 ,  G06F16/483

Abstract: Implementations relate to generating multi-modal response(s) through utilization of large language model(s) (LLM(s)) and other generative model(s). Processor(s) of a system can: receive natural language (NL) based input, generate a multi-modal response that is responsive to the NL based output, and cause the multi-modal response to be rendered. In some implementations, and in generating the multi-modal response, the processor(s) can process, using a LLM, LLM input to generate LLM output, and determine, based on the LLM output, textual content and generative multimedia content for inclusion in the multi-modal response. In some implementations, the generative multimedia content can be generated by another generative model (e.g., an image generator, a video generator, an audio generator, etc.) based on generative multimedia content prompt(s) included in the LLM output and that is indicative of the generative multimedia content. In various implementations, the generative multimedia content can be interleaved between segments of the textual content.

公开(公告)号：US20230297580A1

公开(公告)日：2023-09-21

申请号：US17721873

申请日：2022-04-15

Applicant: Google LLC

Inventor： Sheng Li ,  Garrett Axel Andersen ,  Norman Paul Jouppi ,  Quoc V. Le ,  Liqun Cheng ,  Parthasarathy Ranganathan ,  Julian Paul Grady ,  Yang Li ,  Martin Wicke ,  Yifeng Lu ,  Yun Ni ,  Kun Wang

IPC: G06F16/2457 ,  G06F16/2455 ,  G06N3/063

CPC classification number: G06F16/2457 ,  G06F16/24554 ,  G06N3/063

Abstract: According to various implementations, generally disclosed herein is a hybrid and hierarchical neural architecture search (NAS) approach. The approach includes performing a search space partitioning scheme to divide the search space into sub-search spaces. The approach further includes performing a first type of NAS, such as a Multi-trial NAS, to cover a search across the sub-search spaces. The approach also includes performing a second type of NAS, such as a One-Shot NAS, to cover each sub-search space. The approach further includes automatically stopping the second type of NAS based on one or more early stopping criteria.

公开(公告)号：US20250053751A1

公开(公告)日：2025-02-13

申请号：US18413495

申请日：2024-01-16

Applicant: GOOGLE LLC

Inventor： Oscar Akerlund ,  Evgeny Sluzhaev ,  Golnaz Ghiasi ,  Thang Luong ,  Yifeng Lu ,  Igor Petrovski ,  Agoston Weisz ,  Wei Yu ,  Rakesh Shivanna ,  Michael Andrew Goodman ,  Apoorv Kulshreshtha ,  Yu Du ,  Amin Ghafouri ,  Sanil Jain ,  Dustin Tran ,  Vikas Peswani ,  YaGuang Li

IPC: G06F40/40

Abstract: Implementations relate to generating multi-modal response(s) through utilization of large language model(s) (LLM(s)). Processor(s) of a system can: receive natural language (NL) based input, generate a multi-modal response that is responsive to the NL based output, and cause the multi-modal response to be rendered. In some implementations, and in generating the multi-modal response, the processor(s) can process, using a LLM, LLM input (e.g., that includes at least the NL based input) to generate LLM output, and determine, based on the LLM output, textual content for inclusion in the multi-modal response and multimedia content for inclusion in the multi-modal response. In some implementations, the multimedia content can be obtained based on a multimedia content tag that is included in the LLM output and that is indicative of the multimedia content. In various implementations, the multimedia content can be interleaved between segments of the textual content.

公开(公告)号：US11947923B1

公开(公告)日：2024-04-02

申请号：US18520218

申请日：2023-11-27

Applicant: GOOGLE LLC

Inventor： Sanil Jain ,  Wei Yu ,  Ágoston Weisz ,  Michael Andrew Goodman ,  Diana Avram ,  Amin Ghafouri ,  Golnaz Ghiasi ,  Igor Petrovski ,  Khyatti Gupta ,  Oscar Akerlund ,  Evgeny Sluzhaev ,  Rakesh Shivanna ,  Thang Luong ,  Komal Singh ,  Yifeng Lu ,  Vikas Peswani

IPC: G06F40/40 ,  G06V10/70

CPC classification number: G06F40/40 ,  G06V10/70

Abstract: Implementations relate to managing multimedia content that is obtained by large language model(s) (LLM(s)) and/or generated by other generative model(s). Processor(s) of a system can: receive natural language (NL) based input that requests multimedia content, generate a response that is responsive to the NL based input, and cause the response to be rendered. In some implementations, and in generating the response, the processor(s) can process, using a LLM, LLM input to generate LLM output, and determine, based on the LLM output, at least multimedia content to be included in the response. Further, the processor(s) can evaluate the multimedia content to determine whether it should be included in the response. In response to determining that the multimedia content should not be included in the response, the processor(s) can cause the response, including alternative multimedia content or other textual content, to be rendered.

公开(公告)号：US20230359895A1

公开(公告)日：2023-11-09

申请号：US18313291

申请日：2023-05-05

Applicant: Google LLC

Inventor： Xiangning Chen ,  Chen Liang ,  Da Huang ,  Esteban Alberto Real ,  Yao Liu ,  Kaiyuan Wang ,  Yifeng Lu ,  Quoc V. Le

IPC: G06N3/08

CPC classification number: G06N3/08

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network to perform a machine learning task using a momentum and sign based optimizer.

公开(公告)号：US20250156715A1

公开(公告)日：2025-05-15

申请号：US18839379

申请日：2022-12-27

Applicant: Google LLC

Inventor： Da Huang ,  Chengrun Yang ,  Pieter-Jan Kindermans ,  Hanxiao Liu ,  Quoc V. Le ,  Madeleine Richards Udell ,  Yifeng Lu ,  Gabriel Mintzer Bender

IPC: G06N3/082

Abstract: Provided are neural architecture search techniques that have improved computational efficiency via performance of an initial constraint evaluation and improved gradient update approach. Further, the proposed approaches provide significant improvements for certain modalities of input data, such as tabular datasets.

公开(公告)号：US12277400B1

公开(公告)日：2025-04-15

申请号：US18590498

申请日：2024-02-28

Applicant: GOOGLE LLC

Inventor： Sanil Jain ,  Wei Yu ,  Ágoston Weisz ,  Michael Andrew Goodman ,  Diana Avram ,  Amin Ghafouri ,  Golnaz Ghiasi ,  Igor Petrovski ,  Khyatti Gupta ,  Oscar Akerlund ,  Evgeny Sluzhaev ,  Rakesh Shivanna ,  Thang Luong ,  Komal Singh ,  Yifeng Lu ,  Vikas Peswani

IPC: G06F40/40 ,  G06V10/70

Abstract: Implementations relate to managing multimedia content that is obtained by large language model(s) (LLM(s)) and/or generated by other generative model(s). Processor(s) of a system can: receive natural language (NL) based input that requests multimedia content, generate a response that is responsive to the NL based input, and cause the response to be rendered. In some implementations, and in generating the response, the processor(s) can process, using a LLM, LLM input to generate LLM output, and determine, based on the LLM output, at least multimedia content to be included in the response. Further, the processor(s) can evaluate the multimedia content to determine whether it should be included in the response. In response to determining that the multimedia content should not be included in the response, the processor(s) can cause the response, including alternative multimedia content or other textual content, to be rendered.