公开(公告)号：US12277400B1

公开(公告)日：2025-04-15

申请号：US18590498

申请日：2024-02-28

Applicant: GOOGLE LLC

Inventor： Sanil Jain ,  Wei Yu ,  Ágoston Weisz ,  Michael Andrew Goodman ,  Diana Avram ,  Amin Ghafouri ,  Golnaz Ghiasi ,  Igor Petrovski ,  Khyatti Gupta ,  Oscar Akerlund ,  Evgeny Sluzhaev ,  Rakesh Shivanna ,  Thang Luong ,  Komal Singh ,  Yifeng Lu ,  Vikas Peswani

IPC: G06F40/40 ,  G06V10/70

Abstract: Implementations relate to managing multimedia content that is obtained by large language model(s) (LLM(s)) and/or generated by other generative model(s). Processor(s) of a system can: receive natural language (NL) based input that requests multimedia content, generate a response that is responsive to the NL based input, and cause the response to be rendered. In some implementations, and in generating the response, the processor(s) can process, using a LLM, LLM input to generate LLM output, and determine, based on the LLM output, at least multimedia content to be included in the response. Further, the processor(s) can evaluate the multimedia content to determine whether it should be included in the response. In response to determining that the multimedia content should not be included in the response, the processor(s) can cause the response, including alternative multimedia content or other textual content, to be rendered.

公开(公告)号：US20250013438A1

公开(公告)日：2025-01-09

申请号：US18746997

申请日：2024-06-18

Applicant: Google LLC

Inventor： Michael Andrew Goodman ,  Deepak Goyal

IPC: G06F8/35 ,  G06F8/34 ,  G10L13/08 ,  G10L15/22

Abstract: A method include receiving a natural language prompt from a user comprising a command to generate a code script for an automated assistant to perform a routine. The routine includes multiple discrete actions specified by the natural language prompt. The method further includes processing, by a pre-trained large language model (LLM), the natural language prompt to generate the code script as an LLM output, and processing the code script to determine the code script is incomplete, thereby rendering the code script unsuitable for the automated assistant to fulfill performance of the routine. Based on determining the code script is incomplete, the method includes issuing a user prompt soliciting the user to provide additional information needed to complete the code script and receiving user input of the additional information needed to complete the code script. The method includes supplementing the code script with the additional information to render completed code script.

公开(公告)号：US20240031483A1

公开(公告)日：2024-01-25

申请号：US18375857

申请日：2023-10-02

Applicant: GOOGLE LLC

Inventor： Yuval Baror ,  Michael Andrew Goodman ,  Yoav Tzur

IPC: H04M3/493 ,  G06F16/901 ,  H04M3/51

CPC classification number: H04M3/4936 ,  G06F16/9027 ,  H04M3/5166

Abstract: Implementations are directed to utilizing an assistant to automatically navigate an interactive voice response (IVR) tree to arrive at a target state during an assisted telephone call. The assistant can receive input to initiate the assisted telephone call, identify an entity to engage with, on behalf of the user, and during the assisted telephone call, based on the input, and identify an IVR tree stored in association with the entity. In various implementations, navigation of the IVR tree can be modified based on interaction(s) detected at a client device subsequent to initiating the assisted telephone call. In various implementations, the assisted telephone call can be initiated from a search interface, and the target state can be associated with a given search result. In various implementations, the IVR tree can be dynamic in that only a subset of candidate state(s) of the IVR tree may be available as the target state.

公开(公告)号：US11811969B2

公开(公告)日：2023-11-07

申请号：US17690201

申请日：2022-03-09

Applicant: GOOGLE LLC

Inventor： Yuval Baror ,  Michael Andrew Goodman ,  Yoav Tzur

IPC: H04M3/493 ,  G06F16/901 ,  H04M3/51

CPC classification number: H04M3/4936 ,  G06F16/9027 ,  H04M3/5166

Abstract: Implementations are directed to utilizing an assistant to automatically navigate an interactive voice response (IVR) tree to arrive at a target state during an assisted telephone call. The assistant can receive input to initiate the assisted telephone call, identify an entity to engage with, on behalf of the user, and during the assisted telephone call, based on the input, and identify an IVR tree stored in association with the entity. In various implementations, navigation of the IVR tree can be modified based on interaction(s) detected at a client device subsequent to initiating the assisted telephone call. In various implementations, the assisted telephone call can be initiated from a search interface, and the target state can be associated with a given search result. In various implementations, the IVR tree can be dynamic in that only a subset of candidate state(s) of the IVR tree may be available as the target state.

公开(公告)号：US11303749B1

公开(公告)日：2022-04-12

申请号：US17068511

申请日：2020-10-12

Applicant: Google LLC

Inventor： Yuval Baror ,  Michael Andrew Goodman ,  Yoav Tzur

IPC: H04M3/493 ,  G06F16/901 ,  H04M3/51

Abstract: Implementations are directed to utilizing an assistant to automatically navigate an interactive voice response (IVR) tree to arrive at a target state during an assisted telephone call. The assistant can receive input to initiate the assisted telephone call, identify an entity to engage with, on behalf of the user, and during the assisted telephone call, based on the input, and identify an IVR tree stored in association with the entity. In various implementations, navigation of the IVR tree can be modified based on interaction(s) detected at a client device subsequent to initiating the assisted telephone call. In various implementations, the assisted telephone call can be initiated from a search interface, and the target state can be associated with a given search result. In various implementations, the IVR tree can be dynamic in that only a subset of candidate state(s) of the IVR tree may be available as the target state.

公开(公告)号：US20210065698A1

公开(公告)日：2021-03-04

申请号：US16339238

申请日：2019-02-01

Applicant: Google LLC

Inventor： Nevzat Topcu ,  Michael Andrew Goodman

IPC: G10L15/22 ,  G06F9/451 ,  G06F16/9032 ,  G08B3/10 ,  G06F3/01

Abstract: Implementations set forth herein relate to initializing performance of an automated assistant routine and/or dismissing an alarm pre-emptively according to satisfaction of one or more conditions. A condition can be satisfied by a user acknowledging the alarm when the alarm is going off, or causing the alarm to be dismissed prior to a time at which the alarm was scheduled for. The user can cause the alarm to be dismissed pre-emptively by interacting with the automated assistant prior to the time the alarm was scheduled for and/or interacting with a device, which is known to the automated assistant, prior to the time that the alarm was scheduled for. In this way, actions that cause an alarm to be dismissed can be recognized and used to initialize other processes, such as an automated assistant routine, thereby reducing a number of inputs needed from a user.

公开(公告)号：US10402450B2

公开(公告)日：2019-09-03

申请号：US15593236

申请日：2017-05-11

Applicant: GOOGLE LLC

Inventor： Michael Andrew Goodman ,  Bibo Xu

IPC: G06F17/30 ,  G10L15/22 ,  G06F16/635 ,  G06F3/16 ,  G06F16/332 ,  G06F1/16 ,  G06F1/18 ,  H04L29/06 ,  G06F3/0488 ,  G06F3/0362 ,  G06F3/0354 ,  G06F16/00 ,  G06F16/242 ,  H04R1/32

Abstract: A method at an electronic device with an audio input device and an audio output device includes: receiving through the audio input device a verbal input from a user; transmitting information corresponding to the verbal input to a remote system; receiving from the remote system a response responsive to the verbal input, the response including information in accordance with one or more criteria; and outputting the response through the audio output device.

公开(公告)号：US20250139379A1

公开(公告)日：2025-05-01

申请号：US18385270

申请日：2023-10-30

Applicant: GOOGLE LLC

Inventor： Sanil Jain ,  Wei Yu ,  Alessandro Agostini ,  Agoston Weisz ,  Michael Andrew Goodman ,  Attila Dankovics ,  Elle Chae ,  Evgeny Sluzhaev ,  Amin Ghafouri ,  Golnaz Ghiasi ,  Igor Petrovski ,  Konstantin Shagin ,  Marcelo Menegali ,  Oscar Akerlund ,  Rakesh Shivanna ,  Thang Luong ,  Tiffany Chen ,  Vikas Peswani ,  Yifeng Lu

IPC: G06F40/40 ,  G06F16/483

Abstract: Implementations relate to generating multi-modal response(s) through utilization of large language model(s) (LLM(s)) and other generative model(s). Processor(s) of a system can: receive natural language (NL) based input, generate a multi-modal response that is responsive to the NL based output, and cause the multi-modal response to be rendered. In some implementations, and in generating the multi-modal response, the processor(s) can process, using a LLM, LLM input to generate LLM output, and determine, based on the LLM output, textual content and generative multimedia content for inclusion in the multi-modal response. In some implementations, the generative multimedia content can be generated by another generative model (e.g., an image generator, a video generator, an audio generator, etc.) based on generative multimedia content prompt(s) included in the LLM output and that is indicative of the generative multimedia content. In various implementations, the generative multimedia content can be interleaved between segments of the textual content.

公开(公告)号：US20240176580A1

公开(公告)日：2024-05-30

申请号：US18432921

申请日：2024-02-05

Applicant: GOOGLE LLC

Inventor： Vikram Aggarwal ,  Michael Andrew Goodman

IPC: G06F3/16 ,  G06F9/451 ,  G06Q10/10

CPC classification number: G06F3/167 ,  G06F9/453 ,  G06Q10/10

Abstract: A method includes determining, by an assistant executing at one or more processors, a default group of actions that the assistant is configured to execute in response to receiving a particular audible command. The method includes determining, by the assistant, based on the default group of actions and a user profile associated with a particular user, a custom group of actions that the assistant is configured to execute in response to receiving the particular audible command from the particular user. The method also includes receiving, by the assistant, an indication of the particular audible command, and determining, by the assistant, whether the indication of particular audible command originated from the particular user. The method further includes, responsive to determining that the indication of particular audible command originated from the particular user, executing, by the assistant, each action from the custom group of actions.

公开(公告)号：US11893311B2

公开(公告)日：2024-02-06

申请号：US18096453

申请日：2023-01-12

Applicant: GOOGLE LLC

Inventor： Vikram Aggarwal ,  Michael Andrew Goodman

IPC: G06F3/16 ,  G06F9/451 ,  G06Q10/10

CPC classification number: G06F3/167 ,  G06F9/453 ,  G06Q10/10

Abstract: A method includes determining, by an assistant executing at one or more processors, a default group of actions that the assistant is configured to execute in response to receiving a particular audible command. The method includes determining, by the assistant, based on the default group of actions and a user profile associated with a particular user, a custom group of actions that the assistant is configured to execute in response to receiving the particular audible command from the particular user. The method also includes receiving, by the assistant, an indication of the particular audible command, and determining, by the assistant, whether the indication of particular audible command originated from the particular user. The method further includes, responsive to determining that the indication of particular audible command originated from the particular user, executing, by the assistant, each action from the custom group of actions.