# ADL-MVDR: All deep learning MVDR beamformer for target speech separation

arXiv: https://arxiv.org/abs/2008.06994 · HTML: https://mlsanity.com/paper/2008.06994/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Aug 2020.

## Built on by (most cited first, 20 of 20)

- [Insights Into Deep Non-linear Filters for Improved Multi-channel Speech Enhancement](https://mlsanity.com/paper/2206.13310/) (2022)
- [Hybrid Neural Networks for On-device Directional Hearing](https://mlsanity.com/paper/2112.05893/) (2021)
- [Implicit Neural Spatial Filtering for Multichannel Source Separation in the Waveform Domain](https://mlsanity.com/paper/2206.15423/) (2022)
- [MIMO Self-attentive RNN Beamformer for Multi-speaker Speech Separation](https://mlsanity.com/paper/2104.08450/) (2021)
- [Weakly-supervised Audio Separation via Bi-modal Semantic Similarity](https://mlsanity.com/paper/2404.01740/) (2024)
- [TF-Mamba: A Time-Frequency Network for Sound Source Localization](https://mlsanity.com/paper/2409.05034/) (2024)
- [MIMO-DoAnet: Multi-channel Input and Multiple Outputs DoA Network with Unknown Number of Sound Sources](https://mlsanity.com/paper/2207.07307/) (2022)
- [End-to-end Multichannel Speaker-Attributed ASR: Speaker Guided Decoder and Input Feature Analysis](https://mlsanity.com/paper/2310.10106/) (2023)
- [Deep Neural Mel-Subband Beamformer for In-car Speech Separation](https://mlsanity.com/paper/2211.12590/) (2022)
- [OpenSep: Leveraging Large Language Models with Textual Inversion for Open World Audio Separation](https://mlsanity.com/paper/2409.19270/) (2024)
- [LMFCA-Net: A Lightweight Model for Multi-Channel Speech Enhancement with Efficient Narrow-Band and Cross-Band Attention](https://mlsanity.com/paper/2502.11462/) (2025)
- [Learning Filterbanks for End-to-End Acoustic Beamforming](https://mlsanity.com/paper/2111.04614/) (2021)
- [A Time-domain Real-valued Generalized Wiener Filter for Multi-channel Neural Separation Systems](https://mlsanity.com/paper/2112.03533/) (2021)
- [Iterative Sound Source Localization for Unknown Number of Sources](https://mlsanity.com/paper/2206.12273/) (2022)
- [Guided Speech Enhancement Network](https://mlsanity.com/paper/2303.07486/) (2023)
- [MIMO-DBnet: Multi-channel Input and Multiple Outputs DOA-aware Beamforming Network for Speech Separation](https://mlsanity.com/paper/2212.03401/) (2022)
- [Enhanced Neural Beamformer with Spatial Information for Target Speech Extraction](https://mlsanity.com/paper/2306.15942/) (2023)
- [RIR-SF: Room Impulse Response Based Spatial Feature for Target Speech Recognition in Multi-Channel Multi-Speaker Scenarios](https://mlsanity.com/paper/2311.00146/) (2023)
- [End-to-End Multi-Microphone Speaker Extraction Using Relative Transfer Functions](https://mlsanity.com/paper/2502.06285/) (2025)
- [SpatialEmb: Extract and Encode Spatial Information for 1-Stage Multi-channel Multi-speaker ASR on Arbitrary Microphone Arrays](https://mlsanity.com/paper/2601.18037/) (2026)

## Abstract (arXiv)

Speech separation algorithms are often used to separate the target speech from other interfering sources. However, purely neural network based speech separation systems often cause nonlinear distortion that is harmful for automatic speech recognition (ASR) systems. The conventional mask-based minimum variance distortionless response (MVDR) beamformer can be used to minimize the distortion, but comes with high level of residual noise. Furthermore, the matrix operations (e.g., matrix inversion) involved in the conventional MVDR solution are sometimes numerically unstable when jointly trained with neural networks. In this paper, we propose a novel all deep learning MVDR framework, where the matrix inversion and eigenvalue decomposition are replaced by two recurrent neural networks (RNNs), to resolve both issues at the same time. The proposed method can greatly reduce the residual noise while keeping the target speech undistorted by leveraging on the RNN-predicted frame-wise beamforming weights. The system is evaluated on a Mandarin audio-visual corpus and compared against several state-of-the-art (SOTA) speech separation systems. Experimental results demonstrate the superiority of the proposed method across several objective metrics and ASR accuracy.