公开(公告)号：US20230230275A1

公开(公告)日：2023-07-20

申请号：US18011601

申请日：2021-11-15

Applicant: Google LLC

Inventor： Tsung-Yi Lin ,  Peter Raymond Florence ,  Yen-Chen Lin ,  Jonathan Tilton Barron

IPC: G06T7/70

CPC classification number: G06T7/70 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30244

Abstract: Provided are systems and methods that invert a trained NeRF model, which stores the structure of a scene or object, to estimate the 6D pose from an image taken with a novel view. 6D pose estimation has a wide range of applications, including visual localization and object pose estimation for robot manipulation.

公开(公告)号：US20250144795A1

公开(公告)日：2025-05-08

申请号：US19008421

申请日：2025-01-02

Applicant: Google LLC

Inventor： Peter Raymond Florence ,  Danny Michael Driess ,  Igor Mordatch ,  Andy Zeng ,  Seyed Mohammad Mehdi Sajjadi ,  Klaus Greff

IPC: B25J9/16

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for controlling an agent interacting with an environment. In one aspect, a method comprises: receiving one or more observations of an environment; receiving an input text sequence that describes a task to be performed by a robot in the environment; generating an encoded representation of the input text sequence in an embedding space; generating a corresponding encoded representation of each of the one or more observations in the embedding space; generating a sequence of input tokens that comprises the encoded representation of the input text sequence and the corresponding encoded representation of each observation; processing the sequence of input tokens using a language model neural network to generate an output text sequence that comprises high-level natural language instructions; and determining, from the high-level natural language instructions, one or more actions to be performed by the robot.