# End-to-End Far-Field Speech Recognition with Unified Dereverberation and Beamforming

arXiv: https://arxiv.org/abs/2005.10479 · HTML: https://mlsanity.com/paper/2005.10479/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2020.

## Built on by (most cited first, 9 of 9)

- [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021)
- [Implicit Neural Spatial Filtering for Multichannel Source Separation in the Waveform Domain](https://mlsanity.com/paper/2206.15423/) (2022)
- [Self-Attention Channel Combinator Frontend for End-to-End Multichannel Far-field Speech Recognition](https://mlsanity.com/paper/2109.04783/) (2021)
- [Speech enhancement with frequency domain auto-regressive modeling](https://mlsanity.com/paper/2309.13537/) (2023)
- [Exploiting Single-Channel Speech for Multi-Channel End-to-End Speech Recognition: A Comparative Study](https://mlsanity.com/paper/2203.16757/) (2022)
- [End-to-End Multi-speaker ASR with Independent Vector Analysis](https://mlsanity.com/paper/2204.00218/) (2022)
- [Mutual Learning of Single- and Multi-Channel End-to-End Neural Diarization](https://mlsanity.com/paper/2210.03459/) (2022)
- [Exploring the Integration of Speech Separation and Recognition with Self-Supervised Learning Representation](https://mlsanity.com/paper/2307.12231/) (2023)
- [Run-Time Adaptation of Neural Beamforming for Robust Speech Dereverberation and Denoising](https://mlsanity.com/paper/2410.22805/) (2024)

## Abstract (arXiv)

Despite successful applications of end-to-end approaches in multi-channel speech recognition, the performance still degrades severely when the speech is corrupted by reverberation. In this paper, we integrate the dereverberation module into the end-to-end multi-channel speech recognition system and explore two different frontend architectures. First, a multi-source mask-based weighted prediction error (WPE) module is incorporated in the frontend for dereverberation. Second, another novel frontend architecture is proposed, which extends the weighted power minimization distortionless response (WPD) convolutional beamformer to perform simultaneous separation and dereverberation. We derive a new formulation from the original WPD, which can handle multi-source input, and replace eigenvalue decomposition with the matrix inverse operation to make the back-propagation algorithm more stable. The above two architectures are optimized in a fully end-to-end manner, only using the speech recognition criterion. Experiments on both spatialized wsj1-2mix corpus and REVERB show that our proposed model outperformed the conventional methods in reverberant scenarios.