公开(公告)号：US20220223037A1

公开(公告)日：2022-07-14

申请号：US17149659

申请日：2021-01-14

Applicant: Baidu USA LLC

Inventor： Kecheng XU ,  Hongyi SUN ,  Qi LUO ,  Wei WANG ,  Zejun LIN ,  Wesley REYNOLDS ,  Feng LIU ,  Jiangtao HU ,  Jinghao MIAO

IPC: G08G1/0967 ,  G08G1/01 ,  G08G1/04 ,  G08G1/133 ,  G08G1/0965 ,  G06N3/04

Abstract: According to various embodiments, systems, methods, and mediums for operating an autonomous driving vehicles (ADV) are described. The embodiments use a number of machine learning models to extract features individually from audio data and visual data captured by sensors mounted on the ADV, and then to fuse these extracted features to create a concatenated feature vectors. The concatenated feature vector is provided to a multiplayer perceptron (MLP) as input to generate a detection result related to the presence of an emergency vehicle in the surrounding environment. The detection result can be used by the ADV to take appropriate actions to comply with the local traffic rules.

公开(公告)号：US20220223170A1

公开(公告)日：2022-07-14

申请号：US17248196

申请日：2021-01-13

Applicant: Baidu USA LLC

Inventor： Hongyi SUN ,  Kecheng XU ,  Qi LUO ,  Zejun LIN ,  Wei WANG ,  Wesley REYNOLDS ,  Jiangtao HU ,  Jinghao MIAO

IPC: G10L25/51 ,  H04R1/08 ,  H04R1/40 ,  H04R3/00 ,  G06N20/00 ,  G06N5/04 ,  G05D1/02

Abstract: Systems and methods for sound source detection and localization utilizing an autonomous driving vehicle (ADV) are disclosed. The method includes receiving audio data from a number of audio sensors mounted on the ADV. The audio data comprises sounds captured by the audio sensors and emitted by one or more sound sources. Based on the received audio data, the method further includes determining a number of sound source information. Each sound source information comprises a confidence score associated with an existence of a specific sound. The method further includes generating a data representation to report whether there exists the specific sound within the driving environment of the ADV. The data representation comprises the determined sound source information. The received audio data and the generated data representation are utilized to subsequently train a machine learning algorithm to recognize the specific sound source during autonomous driving of the ADV in real-time.

公开(公告)号：US20220219736A1

公开(公告)日：2022-07-14

申请号：US17149638

申请日：2021-01-14

Applicant: Baidu USA LLC

Inventor： Kecheng XU ,  Hongyi SUN ,  Qi LUO ,  Wei WANG ,  Zejun LIN ,  Wesley REYNOLDS ,  Feng LIU ,  Jiangtao HU ,  Jinghao MIAO

IPC: B60W60/00 ,  H04R1/08 ,  G10L25/51 ,  G10L25/30 ,  G06K9/62 ,  G06K9/00 ,  G06K9/20 ,  H04R1/40 ,  H04R3/00 ,  H04N5/247 ,  B60W10/20 ,  B60W30/18 ,  B60W10/18 ,  G06N3/04 ,  G05B13/02

Abstract: In one embodiment, an emergency vehicle detection system can be provided in the ADV travelling on a road to detect the presence of an emergency vehicle in a surrounding environment of the ADV using both audio data and visual data. The emergency vehicle detection system can use a trained neutral network to independently generate a detection result from the audio data, and use another trained network to independently generate another detection result from the visual data. The emergency vehicle detection system can fuse the two detection results to determine the position and moving direction of the emergency vehicle. The ADV can take appropriate actions in response to the position and moving direction of the emergency vehicle.

公开(公告)号：US20220223169A1

公开(公告)日：2022-07-14

申请号：US17248172

申请日：2021-01-12

Applicant: Baidu USA LLC

Inventor： Qi LUO ,  Kecheng XU ,  Hongyi SUN ,  Wesley REYNOLDS ,  Zejun LIN ,  Wei WANG ,  Yu CAO ,  Weiman LIN

IPC: G10L25/51 ,  G06N20/00 ,  G07C5/08 ,  G08G1/01 ,  G05D1/02

Abstract: Systems and methods for generating labelled audio data and onboard validation of the labelled audio data utilizing an autonomous driving vehicle (ADV) while the ADV is operating within a driving environment are disclosed. The method includes recording a sound emitted by an object within the driving environment of the ADV, and converting the recorded sound into audio samples. The method further includes labelling the audio samples, and refining the labelled audio samples to produce refined labelled audio data. The refined labelled audio data is utilized to subsequently train a machine learning algorithm to recognize a sound source during autonomous driving of the ADV. The method further includes generating a performance profile of the refined labelled audio data based at least on the audio samples, a position of the object, and a relative direction of the object. The position of the object and the relative direction of the object are determined by a perception system of the ADV.

公开(公告)号：US20220222296A1

公开(公告)日：2022-07-14

申请号：US17147342

申请日：2021-01-12

Applicant: Baidu USA LLC

Inventor： Zejun LIN ,  Qi LUO ,  Kecheng XU ,  Hongyi SUN ,  Wesley REYNOLDS ,  Wei WANG

IPC: G06F16/683 ,  G06N20/00 ,  G05D1/00

Abstract: Systems and methods for automatic generation of labelled audio data are disclosed. The method is performed by an autonomous driving system (ADS) of an autonomous driving vehicle (ADV). The method includes recording a sound emitted by an object within a driving environment, and converting the recorded sound into audio data. The method further includes capturing at least one position of the object while the sound is being recorded. The method further includes automatically labelling the audio data using the captured at least one position of the object as an audio label, to generate labelled audio data, where the labelled audio data is utilized to subsequently train a machine learning algorithm to recognize a sound source during autonomous driving of the ADV.