公开(公告)号：US20230352018A1

公开(公告)日：2023-11-02

申请号：US18218333

申请日：2023-07-05

Applicant: GOOGLE LLC

Inventor： Michael Andrew Goodman

IPC: G10L15/22 ,  G10L15/07 ,  G10L15/26

CPC classification number: G10L15/22 ,  G10L15/07 ,  G10L15/26 ,  G10L2015/223 ,  G10L2015/225

Abstract: Recommending an automated assistant action for inclusion in an existing automated assistant routine of a user, where the existing automated assistant routine includes a plurality of preexisting automated assistant actions. If the user confirms the recommendation through affirmative user interface input, the automated assistant action can be automatically added to the existing automated assistant routine. Thereafter, when the automated assistant routine is initialized, the preexisting automated assistant actions of the routine will be performed, as well as the automated assistant action that was automatically added to the routine in response to affirmative user interface input received in response to the recommendation.

公开(公告)号：US11763813B2

公开(公告)日：2023-09-19

申请号：US17243232

申请日：2021-04-28

Applicant: Google LLC

Inventor： Lior Alon ,  Rafael Goldfarb ,  Dekel Auster ,  Dan Rasin ,  Michael Andrew Goodman ,  Trevor Strohman ,  Nino Tasca ,  Valerie Nygaard ,  Jaclyn Konzelmann

IPC: G10L15/22 ,  G06F3/16 ,  G10L15/08 ,  G10L15/18 ,  G10L15/28

CPC classification number: G10L15/22 ,  G06F3/167 ,  G10L15/083 ,  G10L15/1815 ,  G10L15/285 ,  G10L2015/223

Abstract: Implementations described herein relate to reducing latency in automated assistant interactions. In some implementations, a client device can receive audio data that captures a spoken utterance of a user. The audio data can be processed to determine an assistant command to be performed by an automated assistant. The assistant command can be processed, using a latency prediction model, to generate a predicted latency to fulfill the assistant command. Further, the client device (or the automated assistant) can determine, based on the predicted latency, whether to audibly render pre-cached content for presentation to the user prior to audibly rendering content that is responsive to the spoken utterance. The pre-cached content can be tailored to the assistant command and audibly rendered for presentation to the user while the content is being obtained, and the content can be audibly rendered for presentation to the user subsequent to the pre-cached content.

公开(公告)号：US20220358927A1

公开(公告)日：2022-11-10

申请号：US17872465

申请日：2022-07-25

Applicant: Google LLC

Inventor： Michael Andrew Goodman

IPC: G10L15/22 ,  G10L15/07 ,  G10L15/26

Abstract: Recommending an automated assistant action for inclusion in an existing automated assistant routine of a user, where the existing automated assistant routine includes a plurality of preexisting automated assistant actions. If the user confirms the recommendation through affirmative user interface input, the automated assistant action can be automatically added to the existing automated assistant routine. Thereafter, when the automated assistant routine is initialized, the preexisting automated assistant actions of the routine will be performed, as well as the automated assistant action that was automatically added to the routine in response to affirmative user interface input received in response to the recommendation.

公开(公告)号：US10944867B1

公开(公告)日：2021-03-09

申请号：US16947481

申请日：2020-08-03

Applicant: Google LLC

Inventor： Yuval Baror ,  Michael Andrew Goodman ,  Praveen Krishnakumar

IPC: H04M3/42 ,  H04M3/51 ,  G10L15/22 ,  H04M3/436

Abstract: Implementations are directed to using an assistant to initiate automated telephone calls with entities. Some implementations identify an item of interest, identify a group of entities associated with the item, and initiate the calls with the entities. During a given call with a given entity, the assistant can request a status update regarding the item, and determine a temporal delay before initiating another call with the given entity to request a further status update regarding the item based on information received responsive to the request. Other implementations receive a request to perform an action on behalf of a user, identify a group of entities that can perform the action, and initiate a given call with a given entity. During the given call, the assistant can initiate an additional call with an additional entity, and generate notification(s), for the user, based on result(s) of the given call and/or the additional call.

公开(公告)号：US20250053751A1

公开(公告)日：2025-02-13

申请号：US18413495

申请日：2024-01-16

Applicant: GOOGLE LLC

Inventor： Oscar Akerlund ,  Evgeny Sluzhaev ,  Golnaz Ghiasi ,  Thang Luong ,  Yifeng Lu ,  Igor Petrovski ,  Agoston Weisz ,  Wei Yu ,  Rakesh Shivanna ,  Michael Andrew Goodman ,  Apoorv Kulshreshtha ,  Yu Du ,  Amin Ghafouri ,  Sanil Jain ,  Dustin Tran ,  Vikas Peswani ,  YaGuang Li

IPC: G06F40/40

Abstract: Implementations relate to generating multi-modal response(s) through utilization of large language model(s) (LLM(s)). Processor(s) of a system can: receive natural language (NL) based input, generate a multi-modal response that is responsive to the NL based output, and cause the multi-modal response to be rendered. In some implementations, and in generating the multi-modal response, the processor(s) can process, using a LLM, LLM input (e.g., that includes at least the NL based input) to generate LLM output, and determine, based on the LLM output, textual content for inclusion in the multi-modal response and multimedia content for inclusion in the multi-modal response. In some implementations, the multimedia content can be obtained based on a multimedia content tag that is included in the LLM output and that is indicative of the multimedia content. In various implementations, the multimedia content can be interleaved between segments of the textual content.

公开(公告)号：US12051416B2

公开(公告)日：2024-07-30

申请号：US18228948

申请日：2023-08-01

Applicant: GOOGLE LLC

Inventor： Lior Alon ,  Rafael Goldfarb ,  Dekel Auster ,  Dan Rasin ,  Michael Andrew Goodman ,  Trevor Strohman ,  Nino Tasca ,  Valerie Nygaard ,  Jaclyn Konzelmann

IPC: G10L15/22 ,  G06F3/16 ,  G10L15/08 ,  G10L15/18 ,  G10L15/28

CPC classification number: G10L15/22 ,  G06F3/167 ,  G10L15/083 ,  G10L15/1815 ,  G10L15/285 ,  G10L2015/223

Abstract: Implementations described herein relate to reducing latency in automated assistant interactions. In some implementations, a client device can receive audio data that captures a spoken utterance of a user. The audio data can be processed to determine an assistant command to be performed by an automated assistant. The assistant command can be processed, using a latency prediction model, to generate a predicted latency to fulfill the assistant command. Further, the client device (or the automated assistant) can determine, based on the predicted latency, whether to audibly render pre-cached content for presentation to the user prior to audibly rendering content that is responsive to the spoken utterance. The pre-cached content can be tailored to the assistant command and audibly rendered for presentation to the user while the content is being obtained, and the content can be audibly rendered for presentation to the user subsequent to the pre-cached content.

公开(公告)号：US11947923B1

公开(公告)日：2024-04-02

申请号：US18520218

申请日：2023-11-27

Applicant: GOOGLE LLC

Inventor： Sanil Jain ,  Wei Yu ,  Ágoston Weisz ,  Michael Andrew Goodman ,  Diana Avram ,  Amin Ghafouri ,  Golnaz Ghiasi ,  Igor Petrovski ,  Khyatti Gupta ,  Oscar Akerlund ,  Evgeny Sluzhaev ,  Rakesh Shivanna ,  Thang Luong ,  Komal Singh ,  Yifeng Lu ,  Vikas Peswani

IPC: G06F40/40 ,  G06V10/70

CPC classification number: G06F40/40 ,  G06V10/70

Abstract: Implementations relate to managing multimedia content that is obtained by large language model(s) (LLM(s)) and/or generated by other generative model(s). Processor(s) of a system can: receive natural language (NL) based input that requests multimedia content, generate a response that is responsive to the NL based input, and cause the response to be rendered. In some implementations, and in generating the response, the processor(s) can process, using a LLM, LLM input to generate LLM output, and determine, based on the LLM output, at least multimedia content to be included in the response. Further, the processor(s) can evaluate the multimedia content to determine whether it should be included in the response. In response to determining that the multimedia content should not be included in the response, the processor(s) can cause the response, including alternative multimedia content or other textual content, to be rendered.

公开(公告)号：US20230395073A1

公开(公告)日：2023-12-07

申请号：US18233728

申请日：2023-08-14

Applicant: GOOGLE LLC

Inventor： Nevzat Topcu ,  Michael Andrew Goodman

IPC: G10L15/22 ,  G06F9/451 ,  G06F16/9032 ,  G06F3/01 ,  G08B3/10

CPC classification number: G10L15/22 ,  G06F9/453 ,  G10L15/08 ,  G06F3/017 ,  G08B3/10 ,  G06F16/90332

Abstract: Implementations set forth herein relate to initializing performance of an automated assistant routine and/or dismissing an alarm pre-emptively according to satisfaction of one or more conditions. A condition can be satisfied by a user acknowledging the alarm when the alarm is going off, or causing the alarm to be dismissed prior to a time at which the alarm was scheduled for. The user can cause the alarm to be dismissed pre-emptively by interacting with the automated assistant prior to the time the alarm was scheduled for and/or interacting with a device, which is known to the automated assistant, prior to the time that the alarm was scheduled for. In this way, actions that cause an alarm to be dismissed can be recognized and used to initialize other processes, such as an automated assistant routine, thereby reducing a number of inputs needed from a user.

公开(公告)号：US11727930B2

公开(公告)日：2023-08-15

申请号：US17322035

申请日：2021-05-17

Applicant: Google LLC

Inventor： Nevzat Topcu ,  Michael Andrew Goodman

IPC: G10L15/00 ,  G10L15/22 ,  G06F9/451 ,  G06F16/9032 ,  G06F3/01 ,  G08B3/10 ,  G10L15/08

CPC classification number: G10L15/22 ,  G06F3/017 ,  G06F9/453 ,  G06F16/90332 ,  G08B3/10 ,  G10L15/08 ,  G10L2015/223

Abstract: Implementations set forth herein relate to initializing performance of an automated assistant routine and/or dismissing an alarm pre-emptively according to satisfaction of one or more conditions. A condition can be satisfied by a user acknowledging the alarm when the alarm is going off, or causing the alarm to be dismissed prior to a time at which the alarm was scheduled for. The user can cause the alarm to be dismissed pre-emptively by interacting with the automated assistant prior to the time the alarm was scheduled for and/or interacting with a device, which is known to the automated assistant, prior to the time that the alarm was scheduled for. In this way, actions that cause an alarm to be dismissed can be recognized and used to initialize other processes, such as an automated assistant routine, thereby reducing a number of inputs needed from a user.

公开(公告)号：US20230164266A1

公开(公告)日：2023-05-25

申请号：US18100459

申请日：2023-01-23

Applicant: GOOGLE LLC

Inventor： Yuval Baror ,  Michael Andrew Goodman ,  Praveen Krishnakumar

IPC: H04M3/436 ,  H04M3/42 ,  G10L15/22 ,  H04M3/51 ,  H04M3/428 ,  H04M3/493

CPC classification number: H04M3/436 ,  H04M3/42059 ,  G10L15/22 ,  H04M3/5158 ,  H04M3/42042 ,  H04M3/5166 ,  H04M3/4283 ,  H04M3/4936 ,  H04M3/5183 ,  H04M2201/39

Abstract: Implementations are directed to using an assistant to initiate automated telephone calls with entities. Some implementations identify an item of interest, identify a group of entities associated with the item, and initiate the calls with the entities. During a given call with a given entity, the assistant can request a status update regarding the item, and determine a temporal delay before initiating another call with the given entity to request a further status update regarding the item based on information received responsive to the request. Other implementations receive a request to perform an action on behalf of a user, identify a group of entities that can perform the action, and initiate a given call with a given entity. During the given call, the assistant can initiate an additional call with an additional entity, and generate notification(s), for the user, based on result(s) of the given call and/or the additional call.