# Diffusion Buffer: Online Diffusion-based Speech Enhancement with Sub-Second Latency

arXiv: https://arxiv.org/abs/2506.02908 · HTML: https://mlsanity.com/paper/2506.02908/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 17 of its 26 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018)
- [Supervised Speech Separation Based on Deep Learning: An Overview](https://mlsanity.com/paper/1708.07524/) (2017)
- [Speech Enhancement and Dereverberation with Diffusion-based Generative Models](https://mlsanity.com/paper/2208.05830/) (2022)
- [Real Time Speech Enhancement in the Waveform Domain](https://mlsanity.com/paper/2006.12847/) (2020)
- [FIFO-Diffusion: Generating Infinite Videos from Text without Training](https://mlsanity.com/paper/2405.11473/) (2024)
- [Rolling Diffusion Models](https://mlsanity.com/paper/2402.09470/) (2024)
- [Speech Enhancement with Score-Based Generative Models in the Complex STFT Domain](https://mlsanity.com/paper/2203.17004/) (2022)
- [EARS: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberation](https://mlsanity.com/paper/2406.06185/) (2024)
- [An Investigation of Incorporating Mamba for Speech Enhancement](https://mlsanity.com/paper/2405.06573/) (2024)
- [HiFi++: a Unified Framework for Bandwidth Extension and Speech Enhancement](https://mlsanity.com/paper/2203.13086/) (2022)
- [Reducing the Prior Mismatch of Stochastic Differential Equations for Diffusion-based Speech Enhancement](https://mlsanity.com/paper/2302.14748/) (2023)
- [Universal Score-based Speech Enhancement with High Content Preservation](https://mlsanity.com/paper/2406.12194/) (2024)
- [Unsupervised speech enhancement with diffusion-based generative models](https://mlsanity.com/paper/2309.10450/) (2023)
- [An Analysis of the Variance of Diffusion-based Speech Enhancement](https://mlsanity.com/paper/2402.00811/) (2024)

## Built on by (most cited first, 6 of 6)

- [Real-Time Streamable Generative Speech Restoration with Flow Matching](https://mlsanity.com/paper/2512.19442/) (2025)
- [Shortcut Flow Matching for Speech Enhancement: Step-Invariant flows via single stage training](https://mlsanity.com/paper/2509.21522/) (2025)
- [Diffusion Buffer for Online Generative Speech Enhancement](https://mlsanity.com/paper/2510.18744/) (2025)
- [Real-Time Streaming Mel Vocoding with Generative Flow Matching](https://mlsanity.com/paper/2509.15085/) (2025)
- [A Fast Solver for Interpolating Stochastic Differential Equation Diffusion Models for Speech Restoration](https://mlsanity.com/paper/2603.09508/) (2026)
- [Real-time Speech Restoration using Data Prediction Mean Flows](https://mlsanity.com/paper/2605.16251/) (2026)

## Abstract (arXiv)

Diffusion models are a class of generative models that have been recently used for speech enhancement with remarkable success but are computationally expensive at inference time. Therefore, these models are impractical for processing streaming data in real-time. In this work, we adapt a sliding window diffusion framework to the speech enhancement task. Our approach progressively corrupts speech signals through time, assigning more noise to frames close to the present in a buffer. This approach outputs denoised frames with a delay proportional to the chosen buffer size, enabling a trade-off between performance and latency. Empirical results demonstrate that our method outperforms standard diffusion models and runs efficiently on a GPU, achieving an input-output latency in the order of 0.3 to 1 seconds. This marks the first practical diffusion-based solution for online speech enhancement.