公开(公告)号：US20210278523A1

公开(公告)日：2021-09-09

申请号：US17150590

申请日：2021-01-15

Applicant: UATC, LLC

Inventor： Raquel Urtasun ,  Bin Yang ,  Ming Liang ,  Sergio Casas ,  Runsheng Benson Guo

IPC: G01S13/86 ,  G01S17/89 ,  G01S13/58 ,  G06K9/62 ,  G06N20/00

Abstract: Systems and methods for integrating radar and LIDAR data are disclosed. In particular, a computing system can access radar sensor data and LIDAR data for the area around the autonomous vehicle. The computing system can determine, using the one or more machine-learned models, one or more objects in the area of the autonomous vehicle. The computing system can, for a respective object, select a plurality of radar points from the radar sensor data. The computing system can generate a similarity score for each selected radar point. The computing system can generate weight associated with each radar point based on the similarity score. The computing system can calculate predicted velocity for the respective object based on a weighted average of a plurality of velocities associated with the plurality of radar points. The computing system can generate a proposed motion plan based on the predicted velocity for the respective object.

公开(公告)号：US20210149404A1

公开(公告)日：2021-05-20

申请号：US17022923

申请日：2020-09-16

Applicant: UATC, LLC

Inventor： Wenyuan Zeng ,  Shenlong Wang ,  Renjie Liao ,  Yun Chen ,  Bin Yang ,  Raquel Urtasun

IPC: G05D1/02 ,  G05D1/00 ,  G01S17/89 ,  G06T7/20 ,  G06K9/32

Abstract: The present disclosure is directed to generating trajectories using a structured machine-learned model. In particular, a computing system can obtain sensor data for an area around an autonomous vehicle. The computing system can detect one or more objects based on the sensor data. The computing system can determine a plurality of candidate object trajectories for each object in the one or more objects. The computing system can generate, using the plurality of candidate object trajectories as input to one or more machine-learned models, likelihood data for the plurality of candidate object trajectories. The computing system can update the likelihood values for each of the plurality of candidate object trajectories for each respective object in the one or more objects based on the likelihood values associated with candidate object trajectories for other objects in the one or more objects. The computing system can determine a motion plan for the autonomous vehicle.

公开(公告)号：US20240085908A1

公开(公告)日：2024-03-14

申请号：US18513119

申请日：2023-11-17

Applicant: UATC, LLC

Inventor： Raquel Urtasun ,  Mengye Ren ,  Andrei Pokrovsky ,  Bin Yang

IPC: G05D1/00 ,  G01S17/86 ,  G01S17/89 ,  G01S17/931

CPC classification number: G05D1/0088 ,  G01S17/86 ,  G01S17/89 ,  G01S17/931 ,  G05D1/0246

Abstract: The present disclosure provides systems and methods that apply neural networks such as, for example, convolutional neural networks, to sparse imagery in an improved manner. For example, the systems and methods of the present disclosure can be included in or otherwise leveraged by an autonomous vehicle. In one example, a computing system can extract one or more relevant portions from imagery, where the relevant portions are less than an entirety of the imagery. The computing system can provide the relevant portions of the imagery to a machine-learned convolutional neural network and receive at least one prediction from the machine-learned convolutional neural network based at least in part on the one or more relevant portions of the imagery. Thus, the computing system can skip performing convolutions over regions of the imagery where the imagery is sparse and/or regions of the imagery that are not relevant to the prediction being sought.

公开(公告)号：US20240010241A1

公开(公告)日：2024-01-11

申请号：US18240771

申请日：2023-08-31

Applicant: UATC, LLC

Inventor： Lingyun Li ,  Bin Yang ,  Wenyuan Zeng ,  Ming Liang ,  Mengye Ren ,  Sean Segal ,  Raquel Urtasun Sotil

IPC: B60W60/00 ,  B60W50/00 ,  G06N3/044

CPC classification number: B60W60/00274 ,  B60W50/0097 ,  G06N3/044 ,  B60W2554/4045

Abstract: A computing system can input first relative location embedding data into an interaction transformer model and receive, as an output of the interaction transformer model, motion forecast data for actors relative to a vehicle. The computing system can input the motion forecast data into a prediction model to receive respective trajectories for the actors for a current time step and respective projected trajectories for the actors for a subsequent time step. The computing system can generate second relative location embedding data based on the respective projected trajectories from the second time step. The computing system can produce second motion forecast data using the interaction transformer model based on the second relative location embedding. The computing system can determine second respective trajectories for the actors using the prediction model based on the second forecast data.

公开(公告)号：US11860629B2

公开(公告)日：2024-01-02

申请号：US17363986

申请日：2021-06-30

Applicant: UATC, LLC

Inventor： Raquel Urtasun ,  Mengye Ren ,  Andrei Pokrovsky ,  Bin Yang

IPC: G05D1/00 ,  G01S17/89 ,  G01S17/86 ,  G01S17/931 ,  G05D1/02

CPC classification number: G05D1/0088 ,  G01S17/86 ,  G01S17/89 ,  G01S17/931 ,  G05D1/0246

Abstract: The present disclosure provides systems and methods that apply neural networks such as, for example, convolutional neural networks, to sparse imagery in an improved manner. For example, the systems and methods of the present disclosure can be included in or otherwise leveraged by an autonomous vehicle. In one example, a computing system can extract one or more relevant portions from imagery, where the relevant portions are less than an entirety of the imagery. The computing system can provide the relevant portions of the imagery to a machine-learned convolutional neural network and receive at least one prediction from the machine-learned convolutional neural network based at least in part on the one or more relevant portions of the imagery. Thus, the computing system can skip performing convolutions over regions of the imagery where the imagery is sparse and/or regions of the imagery that are not relevant to the prediction being sought.

公开(公告)号：US20220036579A1

公开(公告)日：2022-02-03

申请号：US17388372

申请日：2021-07-29

Applicant: UATC, LLC

Inventor： Ming Liang ,  Wei-Chiu Ma ,  Sivabalan Manivasagam ,  Raquel Urtasun ,  Bin Yang ,  Ze Yang

IPC: G06T7/70 ,  G06K9/00 ,  G06T17/20 ,  G06N20/00

Abstract: Systems and methods for generating simulation data based on real-world dynamic objects are provided. A method includes obtaining two- and three-dimensional data descriptive of a dynamic object in the real world. The two- and three-dimensional information can be provided as an input to a machine-learned model to receive object model parameters descriptive of a pose and shape modification with respect to a three-dimensional template object model. The parameters can represent a three-dimensional dynamic object model indicative of an object pose and an object shape for the dynamic object. The method can be repeated on sequential two- and three-dimensional information to generate a sequence of object model parameters over time. Portions of a sequence of parameters can be stored as simulation data descriptive of a simulated trajectory of a unique dynamic object. The parameters can be evaluated by an objective function to refine the parameters and train the machine-learned model.

公开(公告)号：US20210303922A1

公开(公告)日：2021-09-30

申请号：US17007651

申请日：2020-08-31

Applicant: UATC, LLC

Inventor： James Tu ,  Sivabalan Manivasagam ,  Mengye Ren ,  Ming Liang ,  Bin Yang ,  Raquel Urtasun

IPC: G06K9/62 ,  G06K9/00 ,  G06N20/00 ,  G01S17/42 ,  G01S17/894 ,  G01S17/931

Abstract: Systems and methods for training object detection models using adversarial examples are provided. A method includes obtaining a training scene and identifying a target object within the training scene. The method includes obtaining an adversarial object and generating a modified training scene based on the adversarial object, the target object, and the training scene. The modified training scene includes the training scene modified to include the adversarial object placed on the target object. The modified training scene is input to a machine-learned model configured to detect the training object. A detection score is determined based on whether the training object is detected, and the machine-learned model and the parameters of the adversarial object are trained based on the detection output. The machine-learned model is trained to maximize the detection output. The parameters of the adversarial object are trained to minimize the detection output.

公开(公告)号：US20210278852A1

公开(公告)日：2021-09-09

申请号：US17150987

申请日：2021-01-15

Applicant: UATC, LLC

Inventor： Raquel Urtasun ,  Bob Qingyuan Wei ,  Mengye Ren ,  Wenyuan Zeng ,  Ming Liang ,  Bin Yang

IPC: G05D1/02 ,  G06T17/05 ,  G06K9/62 ,  B60W60/00 ,  G06N20/00

Abstract: Systems and methods for generating attention masks are provided. In particular, a computing system can access sensor data and map data for an area around an autonomous vehicle. The computing system can generate a voxel grid representation of the sensor data and map data. The computing system can generate an attention mask based on the voxel grid representation. The computing system can generate, by using the voxel grid representation and the attention mask as input to a machine-learned model, an attention weighted feature map. The computing system can determine using the attention weighted feature map, a planning cost volume for an area around the autonomous vehicle. The computing system can select a trajectory for the autonomous vehicle based, at least in part, on the planning cost volume.