-
公开(公告)号:US11490218B1
公开(公告)日:2022-11-01
申请号:US17134097
申请日:2020-12-24
申请人: Apple Inc.
摘要: A device for reproducing spatial audio using a machine learning model may include at least one processor configured to receive multiple audio signals corresponding to a sound scene captured by respective microphones of a device. The at least one processor may be further configured to provide the multiple audio signals to a machine learning model, the machine learning model having been trained based at least in part on a target rendering configuration. The at least one processor may be further configured to provide, responsive to providing the multiple audio signals to the machine learning model, multichannel audio signals that comprise a spatial reproduction of the sound scene in accordance with the target rendering configuration.
-
公开(公告)号:US12010490B1
公开(公告)日:2024-06-11
申请号:US18149659
申请日:2023-01-03
申请人: Apple Inc.
发明人: Symeon Delikaris Manias , Mehrez Souden , Ante Jukic , Matthew S. Connolly , Sabine Webel , Ronald J. Guglielmone, Jr.
摘要: An audio renderer can have a machine learning model that jointly processes audio and visual information of an audiovisual recording. The audio renderer can generate output audio channels. Sounds captured in the audiovisual recording and present in the output audio channels are spatially mapped based on the joint processing of the audio and visual information by the machine learning model. Other aspects are described.
-
公开(公告)号:US11849291B2
公开(公告)日:2023-12-19
申请号:US17322539
申请日:2021-05-17
申请人: Apple Inc.
发明人: Mehrez Souden , Jason Wung , Ante Jukic , Ramin Pishehvar , Joshua D. Atkins
IPC分类号: H04R3/04 , H04R3/00 , H04R5/04 , G10L25/78 , G10L21/0216 , G10L21/0208 , H04M9/08
CPC分类号: H04R3/04 , G10L21/0216 , G10L25/78 , H04R3/005 , H04R5/04 , G10L2021/02082 , G10L2021/02166 , H04M9/082
摘要: A plurality of microphone signals can be captured with a plurality of microphones of the device. One or more echo dominant audio signals can be determined based on a pick-up beam directed towards one or more speakers of a playback device. Sound that is emitted from the one or more speakers and sensed by the plurality of microphones can be removed from plurality of microphone signals, by using the one or more echo dominant audio signals as a reference, resulting in clean audio.
-
公开(公告)号:US11532306B2
公开(公告)日:2022-12-20
申请号:US17111132
申请日:2020-12-03
申请人: Apple Inc.
发明人: Yoon Kim , John Bridle , Joshua D. Atkins , Feipeng Li , Mehrez Souden
IPC分类号: G10L15/22 , H04R1/40 , G10L15/08 , G10L15/04 , H04R3/00 , G10L15/30 , G10L15/18 , G10L15/28 , G10L21/0216 , G10L25/51 , H04R27/00
摘要: Systems and processes for operating an intelligent automated assistant are provided. In accordance with one example, a method includes, at an electronic device with one or more processors, memory, and a plurality of microphones, sampling, at each of the plurality of microphones of the electronic device, an audio signal to obtain a plurality of audio signals; processing the plurality of audio signals to obtain a plurality of audio streams; and determining, based on the plurality of audio streams, whether any of the plurality of audio signals corresponds to a spoken trigger. The method further includes, in accordance with a determination that the plurality of audio signals corresponds to the spoken trigger, initiating a session of the digital assistant; and in accordance with a determination that the plurality of audio signals does not correspond to the spoken trigger, foregoing initiating a session of the digital assistant.
-
公开(公告)号:US11341988B1
公开(公告)日:2022-05-24
申请号:US16578802
申请日:2019-09-23
申请人: Apple Inc.
发明人: Ramin Pishehvar , Feiping Li , Ante Jukic , Mehrez Souden , Joshua D. Atkins
摘要: A hybrid machine learning-based and DSP statistical post-processing technique is disclosed for voice activity detection. The hybrid technique may use a DNN model with a small context window to estimate the probability of speech by frames. The DSP statistical post-processing stage operates on the frame-based speech probabilities from the DNN model to smooth the probabilities and to reduce transitions between speech and non-speech states. The hybrid technique may estimate the soft decision on detected speech in each frame based on the smoothed probabilities, generate a hard decision using a threshold, detect a complete utterance that may include brief pauses, and estimate the end point of the utterance. The hybrid voice activity detection technique may incorporate a target directional probability estimator to estimate the direction of the speech source. The DSP statistical post-processing module may use the direction of the speech source to inform the estimates of the voice activity.
-
公开(公告)号:US20210020189A1
公开(公告)日:2021-01-21
申请号:US16516780
申请日:2019-07-19
申请人: Apple Inc.
发明人: Ante Jukic , Mehrez Souden , Joshua D. Atkins
摘要: A learning based system such as a deep neural network (DNN) is disclosed to estimate a distance from a device to a speech source. The deep learning system may estimate the distance of the speech source at each time frame based on speech signals received by a compact microphone array. Supervised deep learning may be used to learn the effect of the acoustic environment on the non-linear mapping between the speech signals and the distance using multi-channel training data. The deep learning system may estimate the direct speech component that contains information about the direct signal propagation from the speech source to the microphone array and the reverberant speech signal that contains the reverberation effect and noise. The deep learning system may extract signal characteristics of the direct signal component and the reverberant signal component and estimate the distance based on the extracted signal characteristics using the learned mapping.
-
公开(公告)号:US11508388B1
公开(公告)日:2022-11-22
申请号:US17100802
申请日:2020-11-20
申请人: Apple Inc.
IPC分类号: G10L21/0232 , H04R1/40 , G10L25/30 , G06N3/08 , H04R3/00 , G10L21/0216
摘要: A device for processing audio signals in a time-domain includes a processor configured to receive multiple audio signals corresponding to respective microphones of at least two or more microphones of the device, at least one of the multiple audio signals comprising speech of a user of the device. The processor is configured to provide the multiple audio signals to a machine learning model, the machine learning model having been trained based at least in part on an expected position of the user of the device and expected positions of the respective microphones on the device. The processor is configured to provide an audio signal that is enhanced with respect to the speech of the user relative to the multiple audio signals, wherein the audio signal is a waveform output from the machine learning model.
-
公开(公告)号:US20220366927A1
公开(公告)日:2022-11-17
申请号:US17321411
申请日:2021-05-15
申请人: Apple Inc.
发明人: Ramin Pishehvar , Ante Jukic , Mehrez Souden , Jason Wung , Feipeng Li , Joshua D. Atkins
IPC分类号: G10L21/0216 , G10L15/16 , G06N20/00
摘要: Disclosed is a multi-task machine learning model such as a time-domain deep neural network (DNN) that jointly generate an enhanced target speech signal and target audio parameters from a mixed signal of target speech and interference signal. The DNN may encode the mixed signal, determine masks used to jointly estimate the target signal and the target audio parameters based on the encoded mixed signal, apply the mask to separate the target speech from the interference signal to jointly estimate the target signal and the target audio parameters, and decode the masked features to enhance the target speech signal and to estimate the target audio parameters. The target audio parameters may include a voice activity detection (VAD) flag of the target speech. The DNN may leverage multi-channel audio signal and multi-modal signals such as video signals of the target speaker to improve the robustness of the enhanced target speech signal.
-
公开(公告)号:US20220059123A1
公开(公告)日:2022-02-24
申请号:US17514694
申请日:2021-10-29
申请人: Apple Inc.
发明人: Jonathan D. Sheaffer , Joshua D. Atkins , Mehrez Souden , Symeon Delikaris Manias , Sean A. Ramprashad
IPC分类号: G10L25/78 , G10L21/0272 , G06T7/246 , H04R3/00
摘要: Processing of ambience and speech can include extracting from audio signals, ambience and speech signals. One or more spatial parameters can be generated that define spatial characteristics of ambience sound in the one or more ambience audio signals. The primary speech signal, the one or more ambience audio signals, and the spatial parameters can be encoded into one or more encoded data streams. Other aspects are described and claimed.
-
公开(公告)号:US11222652B2
公开(公告)日:2022-01-11
申请号:US16516780
申请日:2019-07-19
申请人: Apple Inc.
发明人: Ante Jukic , Mehrez Souden , Joshua D. Atkins
摘要: A learning based system such as a deep neural network (DNN) is disclosed to estimate a distance from a device to a speech source. The deep learning system may estimate the distance of the speech source at each time frame based on speech signals received by a compact microphone array. Supervised deep learning may be used to learn the effect of the acoustic environment on the non-linear mapping between the speech signals and the distance using multi-channel training data. The deep learning system may estimate the direct speech component that contains information about the direct signal propagation from the speech source to the microphone array and the reverberant speech signal that contains the reverberation effect and noise. The deep learning system may extract signal characteristics of the direct signal component and the reverberant signal component and estimate the distance based on the extracted signal characteristics using the learned mapping.
-
-
-
-
-
-
-
-
-