# Partially Adaptive Multichannel Joint Reduction of Ego-noise and Environmental Noise

arXiv: https://arxiv.org/abs/2303.15042 · HTML: https://mlsanity.com/paper/2303.15042/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 32 references that are in the map.
- First posted on arXiv 27 Mar 2023.

## Builds on (most cited first, 9 of 9)

- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [NeMo: a toolkit for building AI applications using Neural Modules](https://mlsanity.com/paper/1909.09577/) (2019)
- [Supervised Speech Separation Based on Deep Learning: An Overview](https://mlsanity.com/paper/1708.07524/) (2017)
- [QuartzNet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutions](https://mlsanity.com/paper/1910.10261/) (2019)
- [A variance modeling framework based on variational autoencoders for speech enhancement](https://mlsanity.com/paper/1902.01605/) (2019)
- [Semi-supervised multichannel speech enhancement with variational autoencoders and non-negative matrix factorization](https://mlsanity.com/paper/1811.06713/) (2018)
- [Enhanced Robot Speech Recognition Using Biomimetic Binaural Sound Source Localization](https://mlsanity.com/paper/1902.05446/) (2019)
- [Phase-Optimized K-SVD for Signal Extraction from Underdetermined Multichannel Sparse Mixtures](https://mlsanity.com/paper/1410.2430/) (2014)

## Built on by (most cited first, 1 of 1)

- [Bring the Noise: Introducing Noise Robustness to Pretrained Automatic Speech Recognition](https://mlsanity.com/paper/2309.02145/) (2023)

## Abstract (arXiv)

Human-robot interaction relies on a noise-robust audio processing module capable of estimating target speech from audio recordings impacted by environmental noise, as well as self-induced noise, so-called ego-noise. While external ambient noise sources vary from environment to environment, ego-noise is mainly caused by the internal motors and joints of a robot. Ego-noise and environmental noise reduction are often decoupled, i.e., ego-noise reduction is performed without considering environmental noise. Recently, a variational autoencoder (VAE)-based speech model has been combined with a fully adaptive non-negative matrix factorization (NMF) noise model to recover clean speech under different environmental noise disturbances. However, its enhancement performance is limited in adverse acoustic scenarios involving, e.g. ego-noise. In this paper, we propose a multichannel partially adaptive scheme to jointly model ego-noise and environmental noise utilizing the VAE-NMF framework, where we take advantage of spatially and spectrally structured characteristics of ego-noise by pre-training the ego-noise model, while retaining the ability to adapt to unknown environmental noise. Experimental results show that our proposed approach outperforms the methods based on a completely fixed scheme and a fully adaptive scheme when ego-noise and environmental noise are present simultaneously.