# Unsupervised Speech Enhancement using Data-defined Priors

arXiv: https://arxiv.org/abs/2509.22942 · HTML: https://mlsanity.com/paper/2509.22942/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 24 references that are in the map.
- First posted on arXiv 26 Sep 2025.

## Builds on (most cited first, 14 of 14)

- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [High-Fidelity Audio Compression with Improved RVQGAN](https://mlsanity.com/paper/2306.06546/) (2023)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022](https://mlsanity.com/paper/2204.02152/) (2022)
- [DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressors](https://mlsanity.com/paper/2010.15258/) (2020)
- [SEGAN: Speech Enhancement Generative Adversarial Network](https://mlsanity.com/paper/1703.09452/) (2017)
- [MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement](https://mlsanity.com/paper/1905.04874/) (2019)
- [A Fully Convolutional Neural Network for Speech Enhancement](https://mlsanity.com/paper/1609.07132/) (2016)
- [MetricGAN-U: Unsupervised speech enhancement/ dereverberation based only on noisy/ reverberated speech](https://mlsanity.com/paper/2110.05866/) (2021)
- [URGENT Challenge: Universality, Robustness, and Generalizability For Speech Enhancement](https://mlsanity.com/paper/2406.04660/) (2024)
- [Noisy-target Training: A Training Strategy for DNN-based Speech Enhancement without Clean Speech](https://mlsanity.com/paper/2101.08625/) (2021)
- [Analysis of Noisy-target Training for DNN-based speech enhancement](https://mlsanity.com/paper/2211.01198/) (2022)

## Abstract (arXiv)

The majority of deep learning-based speech enhancement methods require paired clean-noisy speech data. Collecting such data at scale in real-world conditions is infeasible, which has led the community to rely on synthetically generated noisy speech. However, this introduces a gap between the training and testing phases. In this work, we propose a novel dual-branch encoder-decoder architecture for unsupervised speech enhancement that separates the input into clean speech and residual noise. Adversarial training is employed to impose priors on each branch, defined by unpaired datasets of clean speech and, optionally, noise. Experimental results show that our method achieves performance comparable to leading unsupervised speech enhancement approaches. Furthermore, we demonstrate the critical impact of clean speech data selection on enhancement performance. In particular, our findings reveal that performance may appear overly optimistic when in-domain clean speech data are used for prior definition -- a practice adopted in previous unsupervised speech enhancement studies.