# Mixture to Beamformed Mixture: Leveraging Beamformed Mixture as Weak-Supervision for Speech Enhancement and Noise-Robust ASR

arXiv: https://arxiv.org/abs/2507.15229 · HTML: https://mlsanity.com/paper/2507.15229/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Speech enhancement and noise suppression, one of 11 directions in Speech and audio language models.
- Builds on 31 of its 46 references that are in the map.
- First posted on arXiv 21 Jul 2025.

## Placed in

- Speech and audio language models / [Speech enhancement and noise suppression](https://mlsanity.com/speech/speech-enhancement-and-noise-suppression/)

## Builds on (most cited first, 30 of 31)

- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressors](https://mlsanity.com/paper/2010.15258/) (2020)
- [Supervised Speech Separation Based on Deep Learning: An Overview](https://mlsanity.com/paper/1708.07524/) (2017)
- [CHiME-6 Challenge:Tackling Multispeaker Speech Recognition for Unsegmented Recordings](https://mlsanity.com/paper/2004.09249/) (2020)
- [M2MeT: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challenge](https://mlsanity.com/paper/2110.07393/) (2021)
- [TF-GridNet: Integrating Full- and Sub-Band Modeling for Speech Separation](https://mlsanity.com/paper/2211.12433/) (2022)
- [MIMO-SPEECH: End-to-End Multi-Channel Multi-Speaker Speech Recognition](https://mlsanity.com/paper/1910.06522/) (2019)
- [Far-Field Automatic Speech Recognition](https://mlsanity.com/paper/2009.09395/) (2020)
- [End-to-End Integration of Speech Recognition, Speech Enhancement, and Self-Supervised Learning Representation](https://mlsanity.com/paper/2204.00540/) (2022)
- [RemixIT: Continual self-training of speech enhancement models via bootstrapped remixing](https://mlsanity.com/paper/2202.08862/) (2022)
- [Toward Universal Speech Enhancement for Diverse Input Conditions](https://mlsanity.com/paper/2309.17384/) (2023)
- [Noisy-target Training: A Training Strategy for DNN-based Speech Enhancement without Clean Speech](https://mlsanity.com/paper/2101.08625/) (2021)
- [Convolutive Prediction for Monaural Speech Dereverberation and Noisy-Reverberant Speaker Separation](https://mlsanity.com/paper/2108.07376/) (2021)
- [Finding Strength in Weakness: Learning to Separate Sounds with Weak Supervision](https://mlsanity.com/paper/1911.02182/) (2019)
- [End-to-End Integration of Speech Recognition, Dereverberation, Beamforming, and Self-Supervised Learning Representation](https://mlsanity.com/paper/2210.10742/) (2022)
- [The Multimodal Information based Speech Processing (MISP) 2022 Challenge: Audio-Visual Diarization and Recognition](https://mlsanity.com/paper/2303.06326/) (2023)
- [Unsupervised training of a deep clustering model for multichannel blind source separation](https://mlsanity.com/paper/1904.01340/) (2019)
- [Unsupervised Deep Clustering for Source Separation: Direct Learning from Mixtures using Spatial Information](https://mlsanity.com/paper/1811.01531/) (2018)
- [Adversarial Semi-Supervised Audio Source Separation applied to Singing Voice Extraction](https://mlsanity.com/paper/1711.00048/) (2017)
- [Microphone Array Signal Processing and Deep Learning for Speech Enhancement](https://mlsanity.com/paper/2501.07215/) (2025)
- [30+ Years of Source Separation Research: Achievements and Future Challenges](https://mlsanity.com/paper/2501.11837/) (2025)
- [UNSSOR: Unsupervised Neural Speech Separation by Leveraging Over-determined Training Mixtures](https://mlsanity.com/paper/2305.20054/) (2023)
- [ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Prior](https://mlsanity.com/paper/2505.05657/) (2025)
- [USDnet: Unsupervised Speech Dereverberation via Neural Forward Filtering](https://mlsanity.com/paper/2402.00820/) (2024)
- [Bootstrapping single-channel source separation via unsupervised spatial clustering on stereo mixtures](https://mlsanity.com/paper/1811.02130/) (2018)
- [Improving Design of Input Condition Invariant Speech Enhancement](https://mlsanity.com/paper/2401.14271/) (2024)
- [Overview Of The 2023 Icassp Sp Clarity Challenge: Speech Enhancement For Hearing Aids](https://mlsanity.com/paper/2311.14490/) (2023)
- [SuperM2M: Supervised and Mixture-to-Mixture Co-Learning for Speech Enhancement and Noise-Robust ASR](https://mlsanity.com/paper/2403.10271/) (2024)
- [Mixture to Mixture: Leveraging Close-talk Mixtures as Weak-supervision for Speech Separation](https://mlsanity.com/paper/2402.09313/) (2024)
- [A Survey of Deep Learning for Complex Speech Spectrograms](https://mlsanity.com/paper/2505.08694/) (2025)

## Abstract (arXiv)

In multi-channel speech enhancement and robust automatic speech recognition (ASR), beamforming can typically improve the signal-to-noise ratio (SNR) of the target speaker and produce reliable enhancement with little distortion to target speech. With this observation, we propose to leverage beamformed mixture, which has a higher SNR of the target speaker than the input mixture, as a weak supervision to train deep neural networks (DNNs) to enhance the input mixture. This way, we can train enhancement models using pairs of real-recorded mixture and its beamformed mixture, and potentially realize better generalization to real mixtures, compared with only training the models on simulated mixtures, which usually mismatch real mixtures. Evaluation results on the real-recorded CHiME-4 dataset show the effectiveness of the proposed algorithm.