# A Conformer-based ASR Frontend for Joint Acoustic Echo Cancellation, Speech Enhancement and Speech Separation

arXiv: https://arxiv.org/abs/2111.09935 · HTML: https://mlsanity.com/paper/2111.09935/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2021.

## Built on by (most cited first, 9 of 9)

- [Transformers in Speech Processing: A Survey](https://mlsanity.com/paper/2303.11607/) (2023)
- [SALMONN-omni: A Codec-free LLM for Full-duplex Speech Understanding and Generation](https://mlsanity.com/paper/2411.18138/) (2024)
- [Attentive Temporal Pooling for Conformer-based Streaming Language Identification in Long-form Speech](https://mlsanity.com/paper/2202.12163/) (2022)
- [The RoyalFlush System of Speech Recognition for M2MeT Challenge](https://mlsanity.com/paper/2202.01614/) (2022)
- [Joint Modelling of Spoken Language Understanding Tasks with Integrated Dialog History](https://mlsanity.com/paper/2305.00926/) (2023)
- [Deep Neural Mel-Subband Beamformer for In-car Speech Separation](https://mlsanity.com/paper/2211.12590/) (2022)
- [Improved Long-Form Speech Recognition by Jointly Modeling the Primary and Non-primary Speakers](https://mlsanity.com/paper/2312.11123/) (2023)
- [ERIS: Evolutionary Real-world Interference Scheme for Jailbreaking Audio Large Models](https://mlsanity.com/paper/2509.11128/) (2025)
- [Closing the Gap between Single-User and Multi-User VoiceFilter-Lite](https://mlsanity.com/paper/2202.12169/) (2022)

## Abstract (arXiv)

We present a frontend for improving robustness of automatic speech recognition (ASR), that jointly implements three modules within a single model: acoustic echo cancellation, speech enhancement, and speech separation. This is achieved by using a contextual enhancement neural network that can optionally make use of different types of side inputs: (1) a reference signal of the playback audio, which is necessary for echo cancellation; (2) a noise context, which is useful for speech enhancement; and (3) an embedding vector representing the voice characteristic of the target speaker of interest, which is not only critical in speech separation, but also helpful for echo cancellation and speech enhancement. We present detailed evaluations to show that the joint model performs almost as well as the task-specific models, and significantly reduces word error rate in noisy conditions even when using a large-scale state-of-the-art ASR model. Compared to the noisy baseline, the joint model reduces the word error rate in low signal-to-noise ratio conditions by at least 71% on our echo cancellation dataset, 10% on our noisy dataset, and 26% on our multi-speaker dataset. Compared to task-specific models, the joint model performs within 10% on our echo cancellation dataset, 2% on the noisy dataset, and 3% on the multi-speaker dataset.