# Beyond Performance Plateaus: A Comprehensive Study on Scalability in Speech Enhancement

arXiv: https://arxiv.org/abs/2406.04269 · HTML: https://mlsanity.com/paper/2406.04269/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jun 2024.

## Built on by (most cited first, 5 of 5)

- [A Survey of Deep Learning for Complex Speech Spectrograms](https://mlsanity.com/paper/2505.08694/) (2025)
- [Training-Free Multi-Step Audio Source Separation](https://mlsanity.com/paper/2505.19534/) (2025)
- [UL-UNAS: Ultra-Lightweight U-Nets for Real-Time Speech Enhancement via Network Architecture Search](https://mlsanity.com/paper/2503.00340/) (2025)
- [Comparison of Knowledge Distillation Methods for Low-complexity Multi-microphone Speech Enhancement using the FT-JNF Architecture](https://mlsanity.com/paper/2507.19208/) (2025)
- [TF-MoE: Time-Frequency Mixture-of-Experts for Efficient Speech Separation](https://mlsanity.com/paper/2606.29575/) (2026)

## Abstract (arXiv)

Deep learning-based speech enhancement (SE) models have achieved impressive performance in the past decade. Numerous advanced architectures have been designed to deliver state-of-the-art performance; however, their scalability potential remains unrevealed. Meanwhile, the majority of research focuses on small-sized datasets with restricted diversity, leading to a plateau in performance improvement. In this paper, we aim to provide new insights for addressing the above issues by exploring the scalability of SE models in terms of architectures, model sizes, compute budgets, and dataset sizes. Our investigation involves several popular SE architectures and speech data from different domains. Experiments reveal both similarities and distinctions between the scaling effects in SE and other tasks such as speech recognition. These findings further provide insights into the under-explored SE directions, e.g., larger-scale multi-domain corpora and efficiently scalable architectures.