公开(公告)号：US20240320986A1

公开(公告)日：2024-09-26

申请号：US18734354

申请日：2024-06-05

申请人： NVIDIA Corporation

发明人： Mehmet Kocamaz ,  Neeraj Sajjan ,  Sangmin Oh ,  David Nister ,  Junghyun Kwon ,  Minwoo Park

IPC分类号： G06V20/58 ,  G06N3/08 ,  G06V10/20 ,  G06V10/94 ,  G06V20/56 ,  G06V20/64

CPC分类号： G06V20/58 ,  G06N3/08 ,  G06V10/255 ,  G06V10/95 ,  G06V20/588 ,  G06V20/64

摘要： In various examples, live perception from sensors of an ego-machine may be leveraged to detect objects and assign the objects to bounded regions (e.g., lanes or a roadway) in an environment of the ego-machine in real-time or near real-time. For example, a deep neural network (DNN) may be trained to compute outputs—such as output segmentation masks—that may correspond to a combination of object classification and lane identifiers. The output masks may be post-processed to determine object to lane assignments that assign detected objects to lanes in order to aid an autonomous or semi-autonomous machine in a surrounding environment.

公开(公告)号：US12077190B2

公开(公告)日：2024-09-03

申请号：US16877127

申请日：2020-05-18

申请人： NVIDIA Corporation

发明人： Julia Ng ,  David Nister ,  Zhenyi Zhang ,  Yizhou Wang

IPC分类号： B60W60/00 ,  B60W30/095

CPC分类号： B60W60/00272 ,  B60W30/0953 ,  B60W60/0011 ,  B60W60/0018 ,  B60W2554/4041 ,  B60W2554/4042 ,  B60W2554/80

摘要： In various examples, systems and methods are disclosed for weighting one or more optional paths based on obstacle avoidance or other safety considerations. In some embodiments, the obstacle avoidance considerations may be computed using a comparison of trajectories representative of safety procedures at present and future projected time steps of an ego-vehicle and other actors to ensure that each actor is capable of implementing their respective safety procedure while avoiding collisions at any point along the trajectory. This comparison may include filtering out a path(s) of an actor at a time step(s)—e.g., using a one-dimensional lookup—based on spatial relationships between the actor and the ego-vehicle at the time step(s). Where a particular path—or point along the path—does not satisfy a collision-free standard, the path may be penalized more negatively with respect to the obstacle avoidance considerations, or may be removed from consideration as a potential path.

公开(公告)号：US20240232616A9

公开(公告)日：2024-07-11

申请号：US18343291

申请日：2023-06-28

申请人： NVIDIA Corporation

发明人： Yilin Yang ,  Bala Siva Sashank Jujjavarapu ,  Pekka Janis ,  Zhaoting Ye ,  Sangmin Oh ,  Minwoo Park ,  Daniel Herrera Castro ,  Tommi Koivisto ,  David Nister

IPC分类号： G06N3/08 ,  B60W30/14 ,  B60W60/00 ,  G06F18/214 ,  G06V10/762 ,  G06V20/56

CPC分类号： G06N3/08 ,  B60W30/14 ,  B60W60/0011 ,  G06F18/2155 ,  G06V10/763 ,  G06V20/56

摘要： In various examples, a deep neural network (DNN) is trained to accurately predict, in deployment, distances to objects and obstacles using image data alone. The DNN may be trained with ground truth data that is generated and encoded using sensor data from any number of depth predicting sensors, such as, without limitation, RADAR sensors, LIDAR sensors, and/or SONAR sensors. Camera adaptation algorithms may be used in various embodiments to adapt the DNN for use with image data generated by cameras with varying parameters—such as varying fields of view. In some examples, a post-processing safety bounds operation may be executed on the predictions of the DNN to ensure that the predictions fall within a safety-permissible range.

公开(公告)号：US20240217557A1

公开(公告)日：2024-07-04

申请号：US18602802

申请日：2024-03-12

申请人： NVIDIA Corporation

发明人： Fangkai Yang ,  David Nister ,  Yizhou Wang ,  Rotem Aviv ,  Julia Ng ,  Birgit Henke ,  Hon Leung Lee ,  Yunfei Shi

IPC分类号： B60W60/00 ,  B60W30/18 ,  G08G1/0967

CPC分类号： B60W60/0027 ,  B60W30/18154 ,  B60W30/18159 ,  G08G1/096725 ,  B60W2420/403 ,  B60W2420/408 ,  B60W2552/05

摘要： In various examples, a yield scenario may be identified for a first vehicle. A wait element is received that encodes a first path for the first vehicle to traverse a yield area and a second path for a second vehicle to traverse the yield area. The first path is employed to determine a first trajectory in the yield area for the first vehicle based at least on a first location of the first vehicle at a time and the second path is employed to determine a second trajectory in the yield area for the second vehicle based at least on a second location of the second vehicle at the time. To operate the first vehicle in accordance with a wait state, it may be determined whether there is a conflict between the first trajectory and the second trajectory, where the wait state defines a yielding behavior for the first vehicle.

公开(公告)号：US12001958B2

公开(公告)日：2024-06-04

申请号：US16824199

申请日：2020-03-19

申请人： NVIDIA Corporation

发明人： Alexey Kamenev ,  Nikolai Smolyanskiy ,  Ishwar Kulkarni ,  Ollin Boer Bohan ,  Fangkai Yang ,  Alperen Degirmenci ,  Ruchi Bhargava ,  Urs Muller ,  David Nister ,  Rotem Aviv

IPC分类号： G06N3/088 ,  G06N3/044 ,  G06N3/045

CPC分类号： G06N3/088 ,  G06N3/044 ,  G06N3/045

摘要： In various examples, past location information corresponding to actors in an environment and map information may be applied to a deep neural network (DNN)—such as a recurrent neural network (RNN)—trained to compute information corresponding to future trajectories of the actors. The output of the DNN may include, for each future time slice the DNN is trained to predict, a confidence map representing a confidence for each pixel that an actor is present and a vector field representing locations of actors in confidence maps for prior time slices. The vector fields may thus be used to track an object through confidence maps for each future time slice to generate a predicted future trajectory for each actor. The predicted future trajectories, in addition to tracked past trajectories, may be used to generate full trajectories for the actors that may aid an ego-vehicle in navigating the environment.

公开(公告)号：US20240127454A1

公开(公告)日：2024-04-18

申请号：US18391276

申请日：2023-12-20

申请人： NVIDIA Corporation

发明人： Trung Pham ,  Berta Rodriguez Hervas ,  Minwoo Park ,  David Nister ,  Neda Cvijetic

IPC分类号： G06T7/11 ,  G05B13/02 ,  G06F18/21 ,  G06F18/24 ,  G06N3/04 ,  G06N3/08 ,  G06T3/4046 ,  G06T5/70 ,  G06T11/20 ,  G06V10/26 ,  G06V10/34 ,  G06V10/44 ,  G06V10/82 ,  G06V20/56 ,  G06V30/19 ,  G06V30/262

CPC分类号： G06T7/11 ,  G05B13/027 ,  G06F18/21 ,  G06F18/24 ,  G06N3/04 ,  G06N3/08 ,  G06T3/4046 ,  G06T5/70 ,  G06T11/20 ,  G06V10/267 ,  G06V10/34 ,  G06V10/454 ,  G06V10/82 ,  G06V20/56 ,  G06V30/19173 ,  G06V30/274 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30252 ,  G06T2210/12

摘要： In various examples, live perception from sensors of a vehicle may be leveraged to detect and classify intersection contention areas in an environment of a vehicle in real-time or near real-time. For example, a deep neural network (DNN) may be trained to compute outputs—such as signed distance functions—that may correspond to locations of boundaries delineating intersection contention areas. The signed distance functions may be decoded and/or post-processed to determine instance segmentation masks representing locations and classifications of intersection areas or regions. The locations of the intersections areas or regions may be generated in image-space and converted to world-space coordinates to aid an autonomous or semi-autonomous vehicle in navigating intersections according to rules of the road, traffic priority considerations, and/or the like.

公开(公告)号：US20240101118A1

公开(公告)日：2024-03-28

申请号：US18537527

申请日：2023-12-12

申请人： NVIDIA Corporation

发明人： Sayed Mehdi Sajjadi Mohammadabadi ,  Berta Rodriguez Hervas ,  Hang Dou ,  Igor Tryndin ,  David Nister ,  Minwoo Park ,  Neda Cvijetic ,  Junghyun Kwon ,  Trung Pham

IPC分类号： B60W30/18 ,  B60W30/09 ,  B60W30/095 ,  B60W60/00 ,  G06N3/08 ,  G06V10/25 ,  G06V10/75 ,  G06V10/764 ,  G06V10/80 ,  G06V10/82 ,  G06V20/56 ,  G06V20/70 ,  G08G1/01

CPC分类号： B60W30/18154 ,  B60W30/09 ,  B60W30/095 ,  B60W60/0011 ,  G06N3/08 ,  G06V10/25 ,  G06V10/751 ,  G06V10/764 ,  G06V10/803 ,  G06V10/82 ,  G06V20/56 ,  G06V20/588 ,  G06V20/70 ,  G08G1/0125

摘要： In various examples, live perception from sensors of a vehicle may be leveraged to detect and classify intersections in an environment of a vehicle in real-time or near real-time. For example, a deep neural network (DNN) may be trained to compute various outputs—such as bounding box coordinates for intersections, intersection coverage maps corresponding to the bounding boxes, intersection attributes, distances to intersections, and/or distance coverage maps associated with the intersections. The outputs may be decoded and/or post-processed to determine final locations of, distances to, and/or attributes of the detected intersections.

公开(公告)号：US11927502B2

公开(公告)日：2024-03-12

申请号：US16860824

申请日：2020-04-28

申请人： NVIDIA Corporation

发明人： Jesse Hong ,  Urs Muller ,  Bernhard Firner ,  Zongyi Yang ,  Joyjit Daw ,  David Nister ,  Roberto Giuseppe Luca Valenti ,  Rotem Aviv

IPC分类号： G01M17/007 ,  B60W30/08 ,  B60W30/12 ,  B60W30/14 ,  B60W50/00 ,  B60W50/04 ,  B60W60/00 ,  G06V10/774 ,  G06V20/56 ,  G07C5/08 ,  G06F11/36

CPC分类号： G01M17/007 ,  B60W30/08 ,  B60W30/12 ,  B60W30/143 ,  B60W50/04 ,  B60W50/045 ,  B60W60/0011 ,  G06V10/774 ,  G06V20/56 ,  G07C5/08 ,  B60W2050/0028 ,  G06F11/3684 ,  G06F11/3696

摘要： In various examples, sensor data recorded in the real-world may be leveraged to generate transformed, additional, sensor data to test one or more functions of a vehicle—such as a function of an AEB, CMW, LDW, ALC, or ACC system. Sensor data recorded by the sensors may be augmented, transformed, or otherwise updated to represent sensor data corresponding to state information defined by a simulation test profile for testing the vehicle function(s). Once a set of test data has been generated, the test data may be processed by a system of the vehicle to determine the efficacy of the system with respect to any number of test criteria. As a result, a test set including additional or alternative instances of sensor data may be generated from real-world recorded sensor data to test a vehicle in a variety of test scenarios—including those that may be too dangerous to test in the real-world.

公开(公告)号：US11921502B2

公开(公告)日：2024-03-05

申请号：US18151012

申请日：2023-01-06

申请人： NVIDIA Corporation

发明人： Minwoo Park ,  Xiaolin Lin ,  Hae-Jong Seo ,  David Nister ,  Neda Cvijetic

IPC分类号： G05D1/00 ,  G05D1/02 ,  G06F18/214 ,  G06F18/23 ,  G06F18/2411 ,  G06N3/04 ,  G06N3/08 ,  G06V10/44 ,  G06V10/48 ,  G06V10/75 ,  G06V10/764 ,  G06V10/766 ,  G06V10/776 ,  G06V10/82 ,  G06V10/94 ,  G06V20/56

CPC分类号： G05D1/0077 ,  G05D1/0088 ,  G06F18/2155 ,  G06F18/23 ,  G06F18/2411 ,  G06N3/0418 ,  G06V10/457 ,  G06V10/48 ,  G06V10/751 ,  G06V10/764 ,  G06V10/776 ,  G06V10/82 ,  G06V10/955 ,  G06V20/588 ,  G05D2201/0213

摘要： In various examples, systems and methods are disclosed that preserve rich spatial information from an input resolution of a machine learning model to regress on lines in an input image. The machine learning model may be trained to predict, in deployment, distances for each pixel of the input image at an input resolution to a line pixel determined to correspond to a line in the input image. The machine learning model may further be trained to predict angles and label classes of the line. An embedding algorithm may be used to train the machine learning model to predict clusters of line pixels that each correspond to a respective line in the input image. In deployment, the predictions of the machine learning model may be used as an aid for understanding the surrounding environment—e.g., for updating a world model—in a variety of autonomous machine applications.

公开(公告)号：US20240029447A1

公开(公告)日：2024-01-25

申请号：US18482183

申请日：2023-10-06

申请人： NVIDIA Corporation

发明人： Nikolai SMOLYANSKIY ,  Ryan Oldja ,  Ke Chen ,  Alexander Popov ,  Joachim Pehserl ,  Ibrahim Eden ,  Tilman Wekel ,  David Wehr ,  Ruchi Bhargava ,  David Nister

IPC分类号： G06V20/58 ,  G01S17/931 ,  B60W60/00 ,  G01S17/89 ,  G05D1/00 ,  G06T19/00 ,  G06N3/045

CPC分类号： G06V20/584 ,  G01S17/931 ,  B60W60/0016 ,  B60W60/0027 ,  B60W60/0011 ,  G01S17/89 ,  G05D1/0088 ,  G06T19/006 ,  G06V20/58 ,  G06N3/045 ,  B60W2420/403 ,  G06T2207/10028 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/30261

摘要： A deep neural network(s) (DNN) may be used to detect objects from sensor data of a three dimensional (3D) environment. For example, a multi-view perception DNN may include multiple constituent DNNs or stages chained together that sequentially process different views of the 3D environment. An example DNN may include a first stage that performs class segmentation in a first view (e.g., perspective view) and a second stage that performs class segmentation and/or regresses instance geometry in a second view (e.g., top-down). The DNN outputs may be processed to generate 2D and/or 3D bounding boxes and class labels for detected objects in the 3D environment. As such, the techniques described herein may be used to detect and classify animate objects and/or parts of an environment, and these detections and classifications may be provided to an autonomous vehicle drive stack to enable safe planning and control of the autonomous vehicle.