公开(公告)号：US20250144795A1

公开(公告)日：2025-05-08

申请号：US19008421

申请日：2025-01-02

Applicant: Google LLC

Inventor： Peter Raymond Florence ,  Danny Michael Driess ,  Igor Mordatch ,  Andy Zeng ,  Seyed Mohammad Mehdi Sajjadi ,  Klaus Greff

IPC: B25J9/16

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for controlling an agent interacting with an environment. In one aspect, a method comprises: receiving one or more observations of an environment; receiving an input text sequence that describes a task to be performed by a robot in the environment; generating an encoded representation of the input text sequence in an embedding space; generating a corresponding encoded representation of each of the one or more observations in the embedding space; generating a sequence of input tokens that comprises the encoded representation of the input text sequence and the corresponding encoded representation of each observation; processing the sequence of input tokens using a language model neural network to generate an output text sequence that comprises high-level natural language instructions; and determining, from the high-level natural language instructions, one or more actions to be performed by the robot.