-
公开(公告)号:US12051001B2
公开(公告)日:2024-07-30
申请号:US17972249
申请日:2022-10-24
Applicant: UATC, LLC
Inventor: Raquel Urtasun , Bin Yang , Ming Liang
IPC: G06N3/084 , G01S17/89 , G05D1/00 , G06N20/00 , G06T7/55 , G06T7/73 , G06T11/60 , G06V10/80 , G06V10/82 , G06V20/58 , G06V30/19 , G06V30/24
CPC classification number: G06N3/084 , G01S17/89 , G05D1/0088 , G05D1/0238 , G06N20/00 , G06T7/55 , G06T7/75 , G06T11/60 , G06V10/806 , G06V10/82 , G06V20/58 , G06V30/19173 , G06V30/2504 , G06T2207/10024 , G06T2207/10028 , G06T2207/20081 , G06T2207/20221 , G06T2207/30252
Abstract: Provided are systems and methods that perform multi-task and/or multi-sensor fusion for three-dimensional object detection in furtherance of, for example, autonomous vehicle perception and control. In particular, according to one aspect of the present disclosure, example systems and methods described herein exploit simultaneous training of a machine-learned model ensemble relative to multiple related tasks to learn to perform more accurate multi-sensor 3D object detection. For example, the present disclosure provides an end-to-end learnable architecture with multiple machine-learned models that interoperate to reason about 2D and/or 3D object detection as well as one or more auxiliary tasks. According to another aspect of the present disclosure, example systems and methods described herein can perform multi-sensor fusion (e.g., fusing features derived from image data, light detection and ranging (LIDAR) data, and/or other sensor modalities) at both the point-wise and region of interest (ROI)-wise level, resulting in fully fused feature representations.
-
12.
公开(公告)号:US12013457B2
公开(公告)日:2024-06-18
申请号:US17150590
申请日:2021-01-15
Applicant: UATC, LLC
Inventor: Raquel Urtasun , Bin Yang , Ming Liang , Sergio Casas , Runsheng Benson Guo
IPC: G01S13/86 , G01S7/41 , G01S13/58 , G01S17/89 , G01S17/931 , G06F18/22 , G06F18/2433 , G06F18/25 , G06N3/044 , G06N3/045 , G06N20/00 , G06V10/74 , G06V10/80 , G06V10/84 , G06V20/58
CPC classification number: G01S13/865 , G01S7/417 , G01S13/589 , G01S17/89 , G01S17/931 , G06F18/22 , G06F18/2433 , G06F18/251 , G06N3/045 , G06N20/00 , G06V10/761 , G06V10/803 , G06V10/84 , G06V20/58 , G06N3/044
Abstract: Systems and methods for integrating radar and LIDAR data are disclosed. In particular, a computing system can access radar sensor data and LIDAR data for the area around the autonomous vehicle. The computing system can determine, using the one or more machine-learned models, one or more objects in the area of the autonomous vehicle. The computing system can, for a respective object, select a plurality of radar points from the radar sensor data. The computing system can generate a similarity score for each selected radar point. The computing system can generate weight associated with each radar point based on the similarity score. The computing system can calculate predicted velocity for the respective object based on a weighted average of a plurality of velocities associated with the plurality of radar points. The computing system can generate a proposed motion plan based on the predicted velocity for the respective object.
-
13.
公开(公告)号:US11760386B2
公开(公告)日:2023-09-19
申请号:US17066108
申请日:2020-10-08
Applicant: UATC, LLC
Inventor: Sivabalan Manivasagam , Ming Liang , Bin Yang , Wenyuan Zeng , Raquel Urtasun , Tsun-Hsuan Wang
CPC classification number: B60W60/0027 , G06N3/044 , G06N3/08 , G08G1/0104 , G08G1/0112 , G08G1/22 , H04W4/38 , H04W4/46 , B60W2556/65
Abstract: Systems and methods for vehicle-to-vehicle communications are provided. An example computer-implemented method includes obtaining from a first autonomous vehicle, by a computing system onboard a second autonomous vehicle, a first compressed intermediate environmental representation. The first compressed intermediate environmental representation is indicative of at least a portion of an environment of the second autonomous vehicle and is based at least in part on sensor data acquired by the first autonomous vehicle at a first time. The method includes generating, by the computing system, a first decompressed intermediate environmental representation by decompressing the first compressed intermediate environmental representation. The method includes determining, by the computing system, a first time-corrected intermediate environmental representation based at least in part on the first decompressed intermediate environmental representation. The first time-corrected intermediate environmental representation corresponds to a second time associated with the second autonomous vehicle.
-
公开(公告)号:US11548533B2
公开(公告)日:2023-01-10
申请号:US16826895
申请日:2020-03-23
Applicant: UATC, LLC
Inventor: Ming Liang , Bin Yang , Yun Chen , Raquel Urtasun
Abstract: Systems, methods, tangible non-transitory computer-readable media, and devices associated with object perception and prediction of object motion are provided. For example, a plurality of temporal instance representations can be generated. Each temporal instance representation can be associated with differences in the appearance and motion of objects over past time intervals. Past paths and candidate paths of a set of objects can be determined based on the temporal instance representations and current detections of objects. Predicted paths of the set of objects using a machine-learned model trained that uses the past paths and candidate paths to determine the predicted paths. Past path data that includes information associated with the predicted paths can be generated for each object of the set of objects respectively.
-
公开(公告)号:US11500099B2
公开(公告)日:2022-11-15
申请号:US16353457
申请日:2019-03-14
Applicant: UATC, LLC
Inventor: Ming Liang , Bin Yang , Shenlong Wang , Wei-Chiu Ma , Raquel Urtasun
IPC: G01S17/89 , G06N3/08 , G06N3/04 , G06K9/62 , G01S17/931 , G01S7/481 , G05D1/02 , G06N3/02 , G06V20/64
Abstract: Generally, the disclosed systems and methods implement improved detection of objects in three-dimensional (3D) space. More particularly, an improved 3D object detection system can exploit continuous fusion of multiple sensors and/or integrated geographic prior map data to enhance effectiveness and robustness of object detection in applications such as autonomous driving. In some implementations, geographic prior data (e.g., geometric ground and/or semantic road features) can be exploited to enhance three-dimensional object detection for autonomous vehicle applications. In some implementations, object detection systems and methods can be improved based on dynamic utilization of multiple sensor modalities. More particularly, an improved 3D object detection system can exploit both LIDAR systems and cameras to perform very accurate localization of objects within three-dimensional space relative to an autonomous vehicle. For example, multi-sensor fusion can be implemented via continuous convolutions to fuse image data samples and LIDAR feature maps at different levels of resolution.
-
公开(公告)号:US20220153310A1
公开(公告)日:2022-05-19
申请号:US17528559
申请日:2021-11-17
Applicant: UATC, LLC
Inventor: Bin Yang , Ming Liang , Wenyuan Zeng , Min Bai , Raquel Urtasun
Abstract: Techniques for improving the performance of an autonomous vehicle (AV) by automatically annotating objects surrounding the AV are described herein. A system can obtain sensor data from a sensor coupled to the AV and generate an initial object trajectory for an object using the sensor data. Additionally, the system can determine a fixed value for the object size of the object based on the initial object trajectory. Moreover, the system can generate an updated initial object trajectory, wherein the object size corresponds to the fixed value. Furthermore, the system can determine, based on the sensor data and the updated initial object trajectory, a refined object trajectory. Subsequently, the system can generate a multi-dimensional label for the object based on the refined object trajectory. A motion plan for controlling the AV can be generated based on the multi-dimensional label.
-
公开(公告)号:US20210325882A1
公开(公告)日:2021-10-21
申请号:US17363986
申请日:2021-06-30
Applicant: UATC, LLC
Inventor: Raquel Urtasun , Mengye Ren , Andrei Pokrovsky , Bin Yang
IPC: G05D1/00 , G01S17/89 , G01S17/86 , G01S17/931
Abstract: The present disclosure provides systems and methods that apply neural networks such as, for example, convolutional neural networks, to sparse imagery in an improved manner. For example, the systems and methods of the present disclosure can be included in or otherwise leveraged by an autonomous vehicle. In one example, a computing system can extract one or more relevant portions from imagery, where the relevant portions are less than an entirety of the imagery. The computing system can provide the relevant portions of the imagery to a machine-learned convolutional neural network and receive at least one prediction from the machine-learned convolutional neural network based at least in part on the one or more relevant portions of the imagery. Thus, the computing system can skip performing convolutions over regions of the imagery where the imagery is sparse and/or regions of the imagery that are not relevant to the prediction being sought.
-
公开(公告)号:US20210009166A1
公开(公告)日:2021-01-14
申请号:US16801933
申请日:2020-02-26
Applicant: UATC, LLC
Inventor: Lingyun Li , Bin Yang , Ming Liang , Wenyuan Zeng , Mengye Ren , Sean Segal , Raquel Urtasun
IPC: B60W60/00 , G06N3/08 , G06F16/903
Abstract: A computing system can be configured to input data that describes sensor data into an object detection model and receive, as an output of the object detection model, object detection data describing features of the plurality of the actors relative to the autonomous vehicle. The computing system can generate an input sequence that describes the object detection data. The computing system can analyze the input sequence using an interaction model to produce, as an output of the interaction model, an attention embedding with respect to the plurality of actors. The computing system can be configured to input the attention embedding into a recurrent model and determine respective trajectories for the plurality of actors based on motion forecast data received as an output of the recurrent model.
-
公开(公告)号:US20200160559A1
公开(公告)日:2020-05-21
申请号:US16654487
申请日:2019-10-16
Applicant: UATC, LLC
Inventor: Raquel Urtasun , Bin Yang , Ming Liang
Abstract: Provided are systems and methods that perform multi-task and/or multi-sensor fusion for three-dimensional object detection in furtherance of, for example, autonomous vehicle perception and control. In particular, according to one aspect of the present disclosure, example systems and methods described herein exploit simultaneous training of a machine-learned model ensemble relative to multiple related tasks to learn to perform more accurate multi-sensor 3D object detection. For example, the present disclosure provides an end-to-end learnable architecture with multiple machine-learned models that interoperate to reason about 2D and/or 3D object detection as well as one or more auxiliary tasks. According to another aspect of the present disclosure, example systems and methods described herein can perform multi-sensor fusion (e.g., fusing features derived from image data, light detection and ranging (LIDAR) data, and/or other sensor modalities) at both the point-wise and region of interest (ROI)-wise level, resulting in fully fused feature representations.
-
公开(公告)号:US11780472B2
公开(公告)日:2023-10-10
申请号:US17010416
申请日:2020-09-02
Applicant: UATC, LLC
Inventor: Lingyun Li , Bin Yang , Wenyuan Zeng , Ming Liang , Mengye Ren , Sean Segal , Raquel Urtasun
CPC classification number: B60W60/00272 , B60W60/00276 , G06N20/00 , B60W2554/4049
Abstract: A computing system can input first relative location embedding data into an interaction transformer model and receive, as an output of the interaction transformer model, motion forecast data for actors relative to a vehicle. The computing system can input the motion forecast data into a prediction model to receive respective trajectories for the actors for a current time step and respective projected trajectories for the actors for a subsequent time step. The computing system can generate second relative location embedding data based on the respective projected trajectories from the second time step. The computing system can produce second motion forecast data using the interaction transformer model based on the second relative location embedding. The computing system can determine second respective trajectories for the actors using the prediction model based on the second forecast data.
-
-
-
-
-
-
-
-
-