# Speech enhancement with frequency domain auto-regressive modeling

arXiv: https://arxiv.org/abs/2309.13537 · HTML: https://mlsanity.com/paper/2309.13537/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 79 references that are in the map.
- First posted on arXiv 24 Sep 2023.

## Builds on (most cited first, 21 of 21)

- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [Supervised Speech Separation Based on Deep Learning: An Overview](https://mlsanity.com/paper/1708.07524/) (2017)
- [A Comparative Study on Transformer vs RNN in Speech Applications](https://mlsanity.com/paper/1909.06317/) (2019)
- [Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation](https://mlsanity.com/paper/1910.06379/) (2019)
- [DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement](https://mlsanity.com/paper/2008.00264/) (2020)
- [FullSubNet: A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech Enhancement](https://mlsanity.com/paper/2010.15508/) (2020)
- [Far-Field Automatic Speech Recognition](https://mlsanity.com/paper/2009.09395/) (2020)
- [Voices Obscured in Complex Environmental Settings (VOICES) corpus](https://mlsanity.com/paper/1804.05053/) (2018)
- [End-to-End Multi-Channel Transformer for Speech Recognition](https://mlsanity.com/paper/2102.03951/) (2021)
- [Insights Into Deep Non-linear Filters for Improved Multi-channel Speech Enhancement](https://mlsanity.com/paper/2206.13310/) (2022)
- [The VOiCES from a Distance Challenge 2019 Evaluation Plan](https://mlsanity.com/paper/1902.10828/) (2019)
- [Speech Dereverberation with A Reverberation Time Shortening Target](https://mlsanity.com/paper/2204.08765/) (2022)
- [Deep Speaker Embeddings for Far-Field Speaker Recognition on Short Utterances](https://mlsanity.com/paper/2002.06033/) (2020)
- [End-to-End Far-Field Speech Recognition with Unified Dereverberation and Beamforming](https://mlsanity.com/paper/2005.10479/) (2020)
- [Streaming end-to-end speech recognition with jointly trained neural feature enhancement](https://mlsanity.com/paper/2105.01254/) (2021)
- [An Investigation of End-to-End Multichannel Speech Recognition for Reverberant and Mismatch Conditions](https://mlsanity.com/paper/1904.09049/) (2019)
- [Attention-based ASR with Lightweight and Dynamic Convolutions](https://mlsanity.com/paper/1912.11793/) (2019)
- [End-to-End Speech Recognition With Joint Dereverberation Of Sub-Band Autoregressive Envelopes](https://mlsanity.com/paper/2108.03975/) (2021)
- [Speech Dereverberation with Context-aware Recurrent Neural Networks](https://mlsanity.com/paper/1711.06309/) (2017)
- [Unsupervised Neural Mask Estimator For Generalized Eigen-Value Beamforming Based ASR](https://mlsanity.com/paper/1911.12617/) (2019)
- [Dereverberation of Autoregressive Envelopes for Far-field Speech Recognition](https://mlsanity.com/paper/2108.05520/) (2021)

## Built on by (most cited first, 3 of 3)

- [A Composite Predictive-Generative Approach to Monaural Universal Speech Enhancement](https://mlsanity.com/paper/2505.24576/) (2025)
- [SenSE: Semantic-Aware High-Fidelity Universal Speech Enhancement](https://mlsanity.com/paper/2509.24708/) (2025)
- [Dual-View Predictive Diffusion: Lightweight Speech Enhancement via Spectrogram-Image Synergy](https://mlsanity.com/paper/2602.00568/) (2026)

## Abstract (arXiv)

Speech applications in far-field real world settings often deal with signals that are corrupted by reverberation. The task of dereverberation constitutes an important step to improve the audible quality and to reduce the error rates in applications like automatic speech recognition (ASR). We propose a unified framework of speech dereverberation for improving the speech quality and the ASR performance using the approach of envelope-carrier decomposition provided by an autoregressive (AR) model. The AR model is applied in the frequency domain of the sub-band speech signals to separate the envelope and carrier parts. A novel neural architecture based on dual path long short term memory (DPLSTM) model is proposed, which jointly enhances the sub-band envelope and carrier components. The dereverberated envelope-carrier signals are modulated and the sub-band signals are synthesized to reconstruct the audio signal back. The DPLSTM model for dereverberation of envelope and carrier components also allows the joint learning of the network weights for the down stream ASR task. In the ASR tasks on the REVERB challenge dataset as well as on the VOiCES dataset, we illustrate that the joint learning of speech dereverberation network and the E2E ASR model yields significant performance improvements over the baseline ASR system trained on log-mel spectrogram as well as other benchmarks for dereverberation (average relative improvements of 10-24% over the baseline system). The speech quality improvements, evaluated using subjective listening tests, further highlight the improved quality of the reconstructed audio.