PhASE-Flow: Phonetic-Conditioned Acoustic Flow Matching in SSL Representation Domain for Speech Enhancement
On the map: Diffusion models → Audio, speech and music
Abstract (arXiv)
Flow matching (FM) enables high-fidelity generation, while self-supervised learning (SSL) speech models provide hierarchical representations spanning acoustic and phonetic levels. However, existing FM-based speech enhancement (SE) methods operate primarily in the spectral domain, treating SSL features only as external conditions rather than modeling directly in the SSL latent space. To fully exploit the structural richness of SSL representations, we propose PhASE-Flow, an FM-based SE framework that operates entirely in the SSL space. It models the conditional distribution of clean acoustic representations given phonetic ones, reconstructing the waveform via a neural vocoder. Experiments show that PhASE-Flow outperforms state-of-the-art baselines in perceptual quality and intelligibility. Notably, it achieves competitive performance with only four sampling steps, enabling highly efficient inference. Audio demos are available at https://anonymous.4open.science/w/phase-flow_demo-E6E1/.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 35 of its 43 references that are in the map. First posted on arXiv 16 Jun 2026.
Builds on
35 of 43 references.
Closest ancestors · 6
Same direction, newest first
- 2026Rethinking Flow and Diffusion Bridge Models for Speech Enhancement20 Feb
- 2025SenSE: Semantic-Aware High-Fidelity Universal Speech Enhancement29 Sep
- 2025FlowSE: Efficient and High-Quality Speech Enhancement via Flow Matching26 May
- 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching9 Oct
- 2022StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberation22 Dec
- 2022Speech Enhancement and Dereverberation with Diffusion-based Generative Models11 Aug
Other references · 29
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2022A ConvNet for the 2020scited by 3,579
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
- 2023High-Fidelity Audio Compression with Improved RVQGANcited by 431
- 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
- 2020FSD50K: An Open Dataset of Human-Labeled Sound Eventscited by 342
- 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 298
- 2016FMA: A Dataset For Music Analysiscited by 207
- 2021DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressorscited by 172
- 2019WHAM!: Extending Speech Separation to Noisy Environmentscited by 156
17 more
- 2023Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesiscited by 153
- 2020The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Resultscited by 133
- 2024WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modelingcited by 129
- 2021Simple and Effective Zero-shot Cross-lingual Phoneme Recognitioncited by 79
- 2024mHuBERT-147: A Compact Multilingual HuBERT Modelcited by 66
- 2023ICASSP 2023 Deep Noise Suppression Challengecited by 65
- 2024EARS: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberationcited by 64
- 2022TF-GridNet: Integrating Full- and Sub-Band Modeling for Speech Separationcited by 63
- 2024SpeechBERTScore: Reference-Aware Automatic Evaluation of Speech Generation Leveraging NLP Evaluation Metricscited by 38
- 2023SELM: Speech Enhancement Using Discrete Tokens and Language Modelscited by 36
- 2022Investigating self-supervised learning for speech enhancement and separationcited by 35
- 2022Boosting Self-Supervised Embeddings for Speech Enhancementcited by 24
- 2025AnyEnhance: A Unified Generative Model with Prompt-Guidance and Self-Critic for Voice Enhancementcited by 23
- 2025LLaSE-G1: Incentivizing Generalization Capability for LLaMA-based Speech Enhancementcited by 19
- 2024Generative Speech Foundation Model Pretraining for High-Quality Speech Extraction and Restorationcited by 16
- 2025PASE: Leveraging the Phonological Prior of WavLM for Low-Hallucination Generative Speech Enhancementcited by 5
- 2025Efficient Speech Enhancement via Embeddings from Pre-trained Generative Audioencoderscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Investigating training objective for flow matching-based speech enhancement
- 2025Shortcut Flow Matching for Speech Enhancement: Step-Invariant flows via single stage training
- 2026DiT-Flow: Speech Enhancement Robust to Multiple Distortions based on Flow Matching in Latent Space and Diffusion Transformers
- 2026Beyond U-Net: A Latent-Representation-Aligned Skip-Free Backbone for Flow-Matching Speech Enhancement
- 2026Latent-Level Enhancement with Flow Matching for Robust Automatic Speech Recognition
- 2025Flowing Straighter with Conditional Flow Matching for Accurate Speech Enhancement
- 2025MeanFlowSE: One-Step Generative Speech Enhancement via MeanFlow
- 2026P2Flow: Phoneme-aware Progressive Flow Matching for Extreme Speech Super-Resolution