# A Conformer-based Waveform-domain Neural Acoustic Echo Canceller Optimized for ASR Accuracy

arXiv: https://arxiv.org/abs/2205.03481 · HTML: https://mlsanity.com/paper/2205.03481/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Speech enhancement and noise suppression, one of 11 directions in Speech and audio language models.
- Builds on 18 of its 29 references that are in the map.
- First posted on arXiv 6 May 2022.

## Placed in

- Speech and audio language models / [Speech enhancement and noise suppression](https://mlsanity.com/speech/speech-enhancement-and-noise-suppression/)

## Builds on (most cited first, 18 of 18)

- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [TasNet: time-domain audio separation network for real-time, single-channel speech separation](https://mlsanity.com/paper/1711.00541/) (2017)
- [A Streaming On-Device End-to-End Model Surpassing Server-Side Conventional Model Quality and Latency](https://mlsanity.com/paper/2003.12710/) (2020)
- [Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling](https://mlsanity.com/paper/1902.08295/) (2019)
- [A Better and Faster End-to-End Model for Streaming ASR](https://mlsanity.com/paper/2011.10798/) (2020)
- [DF-Conformer: Integrated architecture of Conv-TasNet and Conformer using linear complexity self-attention for speech enhancement](https://mlsanity.com/paper/2106.15813/) (2021)
- [A Conformer-based ASR Frontend for Joint Acoustic Echo Cancellation, Speech Enhancement and Speech Separation](https://mlsanity.com/paper/2111.09935/) (2021)
- [Acoustic echo cancellation with the dual-signal transformation LSTM network](https://mlsanity.com/paper/2010.14337/) (2020)
- [A Neural Acoustic Echo Canceller Optimized Using An Automatic Speech Recognizer And Large Scale Synthetic Data](https://mlsanity.com/paper/2106.00856/) (2021)
- [End-to-End Complex-Valued Multidilated Convolutional Neural Network for Joint Acoustic Echo Cancellation and Noise Suppression](https://mlsanity.com/paper/2110.00745/) (2021)
- [Cross-attention conformer for context modeling in speech enhancement for ASR](https://mlsanity.com/paper/2111.00127/) (2021)
- [Implicit Acoustic Echo Cancellation for Keyword Spotting and Device-Directed Speech Detection](https://mlsanity.com/paper/2111.10639/) (2021)
- [Nonlinear Residual Echo Suppression Based on Multi-stream Conv-TasNet](https://mlsanity.com/paper/2005.07631/) (2020)
- [EchoFilter: End-to-End Neural Network for Acoustic Echo Cancellation](https://mlsanity.com/paper/2105.14666/) (2021)
- [Mask scalar prediction for improving robust automatic speech recognition](https://mlsanity.com/paper/2204.12092/) (2022)
- [Multi-Scale Attention Neural Network for Acoustic Echo Cancellation](https://mlsanity.com/paper/2106.00010/) (2021)

## Abstract (arXiv)

Acoustic Echo Cancellation (AEC) is essential for accurate recognition of queries spoken to a smart speaker that is playing out audio. Previous work has shown that a neural AEC model operating on log-mel spectral features (denoted "logmel" hereafter) can greatly improve Automatic Speech Recognition (ASR) accuracy when optimized with an auxiliary loss utilizing a pre-trained ASR model encoder. In this paper, we develop a conformer-based waveform-domain neural AEC model inspired by the "TasNet" architecture. The model is trained by jointly optimizing Negative Scale-Invariant SNR (SISNR) and ASR losses on a large speech dataset. On a realistic rerecorded test set, we find that cascading a linear adaptive AEC and a waveform-domain neural AEC is very effective, giving 56-59% word error rate (WER) reduction over the linear AEC alone. On this test set, the 1.6M parameter waveform-domain neural AEC also improves over a larger 6.5M parameter logmel-domain neural AEC model by 20-29% in easy to moderate conditions. By operating on smaller frames, the waveform neural model is able to perform better at smaller sizes and is better suited for applications where memory is limited.