公开(公告)号：US20230311335A1

公开(公告)日：2023-10-05

申请号：US18128953

申请日：2023-03-30

Applicant: GOOGLE LLC

Inventor： Karol Hausman ,  Brian Ichter ,  Sergey Levine ,  Alexander Toshev ,  Fei Xia ,  Carolina Parada

IPC: B25J13/00 ,  B25J11/00 ,  B25J9/16 ,  G06F40/40

CPC classification number: B25J13/003 ,  B25J11/0005 ,  B25J9/163 ,  B25J9/161 ,  G06F40/40

Abstract: Implementations process, using a large language model, a free-form natural language (NL) instruction to generate to generate LLM output. Those implementations generate, based on the LLM output and a NL skill description of a robotic skill, a task-grounding measure that reflects a probability of the skill description in the probability distribution of the LLM output. Those implementations further generate, based on the robotic skill and current environmental state data, a world-grounding measure that reflects a probability of the robotic skill being successful based on the current environmental state data. Those implementations further determine, based on both the task-grounding measure and the world-grounding measure, whether to implement the robotic skill.

公开(公告)号：US11701773B2

公开(公告)日：2023-07-18

申请号：US16622181

申请日：2018-12-04

Applicant: Google LLC

Inventor： Alexander Toshev ,  Fereshteh Sadeghi ,  Sergey Levine

IPC: G05B19/04 ,  G05B19/18 ,  B25J9/16 ,  G05B13/02 ,  G06N3/084 ,  G06N3/044 ,  G06N3/045

CPC classification number: B25J9/163 ,  B25J9/1697 ,  G05B13/027 ,  G06N3/044 ,  G06N3/045 ,  G06N3/084 ,  G05B2219/33056 ,  G05B2219/39391 ,  G05B2219/40499 ,  G05B2219/42152

Abstract: Training and/or using a recurrent neural network model for visual servoing of an end effector of a robot. In visual servoing, the model can be utilized to generate, at each of a plurality of time steps, an action prediction that represents a prediction of how the end effector should be moved to cause the end effector to move toward a target object. The model can be viewpoint invariant in that it can be utilized across a variety of robots having vision components at a variety of viewpoints and/or can be utilized for a single robot even when a viewpoint, of a vision component of the robot, is drastically altered. Moreover, the model can be trained based on a large quantity of simulated data that is based on simulator(s) performing simulated episode(s) in view of the model. One or more portions of the model can be further trained based on a relatively smaller quantity of real training data.

公开(公告)号：US20220305647A1

公开(公告)日：2022-09-29

申请号：US17638469

申请日：2019-08-27

Applicant: GOOGLE LLC

Inventor： Anthony Jacob Piergiovanni ,  Anelia Angelova ,  Alexander Toshev ,  Michael Ryoo

IPC: B25J9/16 ,  B25J19/02 ,  G06V40/10

Abstract: Techniques are disclosed that enable the generation of predicted sequences of terminals using a generator model portion of a prediction model. Various implementations include controlling actuators of a robot based on the predicted sequences of terminals. Additional or alternative implementations include jointly training the generator model portion of the prediction model using a discriminator model portion of the prediction model using, for example, stochastic adversarial based sampling.