Multi-attention machine learning for object detection and classification

    公开(公告)号:US12299997B1

    公开(公告)日:2025-05-13

    申请号:US17952604

    申请日:2022-09-26

    Applicant: Zoox, Inc.

    Abstract: Techniques for detecting, locating, and/or classifying objects based on multiple sensor data inputs received from different sensor modalities. The techniques may include receiving sensor data generated by different sensor modalities of a vehicle, the sensor data including at least first sensor data generated by a first sensor modality and second sensor data generated by a second sensor modality. In some examples, the sensor data may be input into a machine-learning pipeline. The machine-learning pipeline may be configured to determine locations of objects in an environment surrounding the vehicle based at least in part on a correlation, by the multi-attention component, of the first sensor data and the second sensor data. The techniques may also include receiving, from the machine-learning pipeline, an output indicating a location of an object in the environment.

    Emergency sound localization
    6.
    发明授权

    公开(公告)号:US11516613B1

    公开(公告)日:2022-11-29

    申请号:US17077746

    申请日:2020-10-22

    Applicant: Zoox, Inc.

    Abstract: Techniques for determining information associated with sounds detected in an environment based on audio data are discussed herein. Audio sensors of a vehicle may determine audio data associated with sounds from the environment. Sounds may be caused by objects in the environment such as emergency vehicles, construction zones, non-emergency vehicles, humans, audio speakers, nature, etc. A model may determine a classification of the audio data and/or a probability value representing a likelihood that sound in the audio data is associated with the classification. A direction of arrival may be determined based on receiving classification values from multiple audio sensors of the vehicle, and other actions can be performed or the vehicle can be controlled based on the direction of arrival.

    Emergency vehicle detection
    7.
    发明授权

    公开(公告)号:US11322019B2

    公开(公告)日:2022-05-03

    申请号:US16661898

    申请日:2019-10-23

    Applicant: Zoox, Inc.

    Abstract: Techniques for determining a direction of arrival of an emergency are discussed. A plurality of audio sensors of a vehicle can receive audio data associated with the vehicle. An audio sensor pair can be selected from the plurality of audio sensors to generate audio data representing sound in an environment of the vehicle. An angular spectrum associated with the audio sensor pair can be determined based on the audio data. A feature associated with the audio data can be determined based on the angular spectrum and/or the audio data itself. A direction of arrival (DoA) value associated with the sound may be determined based on the feature using a machine learned model. An emergency sound (e.g., a siren) can be detected in the audio data and a direction associated with the emergency relative to the vehicle can be determined based on the feature and the DoA value.

    EMERGENCY VEHICLE DETECTION
    8.
    发明申请

    公开(公告)号:US20210125494A1

    公开(公告)日:2021-04-29

    申请号:US16661898

    申请日:2019-10-23

    Applicant: Zoox, Inc.

    Abstract: Techniques for determining a direction of arrival of an emergency are discussed. A plurality of audio sensors of a vehicle can receive audio data associated with the vehicle. An audio sensor pair can be selected from the plurality of audio sensors to generate audio data representing sound in an environment of the vehicle. An angular spectrum associated with the audio sensor pair can be determined based on the audio data. A feature associated with the audio data can be determined based on the angular spectrum and/or the audio data itself. A direction of arrival (DoA) value associated with the sound may be determined based on the feature using a machine learned model. An emergency sound (e.g., a siren) can be detected in the audio data and a direction associated with the emergency relative to the vehicle can be determined based on the feature and the DoA value.

Patent Agency Ranking