UniPASE: A Generative Model for Universal Speech Enhancement with High Fidelity and Low Hallucinations
Not on the map. Why?
Abstract (arXiv)
Universal speech enhancement (USE) aims to restore speech signals from diverse distortions across multiple sampling rates. We propose UniPASE, an extension of the low-hallucination PASE framework tailored for USE. At its core is DeWavLM-Omni, a unified representation-level enhancement module fine-tuned from WavLM via knowledge distillation on a large-scale supervised multi-distortion dataset. This module directly converts degraded waveforms into clean and linguistically faithful phonetic representations, ensuring robust enhancement with minimal linguistic hallucination. Based on these enhanced phonetic representations, an Adapter generates enhanced acoustic representations containing rich acoustic details, which a neural Vocoder uses to reconstruct corresponding high-fidelity 16-kHz waveforms. A PostNet then converts the waveforms to 48~kHz before resampling them to their original rates, enabling seamless handling of inputs and outputs at multiple sampling rates. Experimental results on several evaluation datasets, covering sub-tasks and full tasks, demonstrate that UniPASE achieves superior or competitive performance compared with existing state-of-the-art models. The proposed model also serves as the backbone of our submission to the URGENT 2026 Challenge, which achieved 1st place in the objective evaluation. The source code and audio demos are available at https://github.com/xiaobin-rong/unipase/.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 64 of its 81 references that are in the map. First posted on arXiv 16 Apr 2026.
Builds on
64 of 81 references.
References · 64
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
- 2022A ConvNet for the 2020scited by 3,579
- 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
- 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
- 2016Least Squares Generative Adversarial Networkscited by 1,001
- 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
- 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 476
- 2023High-Fidelity Audio Compression with Improved RVQGANcited by 431
- 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
- 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 347
38 more
- 2020FSD50K: An Open Dataset of Human-Labeled Sound Eventscited by 342
- 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 231
- 2016FMA: A Dataset For Music Analysiscited by 207
- 2021NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasetscited by 192
- 2021DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressorscited by 172
- 2022Comparative layer-wise analysis of self-supervised speech modelscited by 158
- 2019WHAM!: Extending Speech Separation to Noisy Environmentscited by 156
- 2023Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesiscited by 153
- 2020The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Resultscited by 133
- 2024WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modelingcited by 129
- 2022StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberationcited by 105
- 2021Simple and Effective Zero-shot Cross-lingual Phoneme Recognitioncited by 79
- 2022Universal Speech Enhancement with Score-based Diffusioncited by 75
- 2024OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformercited by 74
- 2024Self-Supervised Speech Representations are More Phonetic than Semanticcited by 71
- 2024mHuBERT-147: A Compact Multilingual HuBERT Modelcited by 66
- 2023ICASSP 2023 Deep Noise Suppression Challengecited by 65
- 2024EARS: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberationcited by 64
- 2022TF-GridNet: Integrating Full- and Sub-Band Modeling for Speech Separationcited by 63
- 2022Pushing the limits of raw waveform speaker recognitioncited by 51
- 2024SpeechBERTScore: Reference-Aware Automatic Evaluation of Speech Generation Leveraging NLP Evaluation Metricscited by 38
- 2022VoiceFixer: A Unified Framework for High-Fidelity Speech Restorationcited by 37
- 2023SELM: Speech Enhancement Using Discrete Tokens and Language Modelscited by 36
- 2022Investigating self-supervised learning for speech enhancement and separationcited by 35
- 2022Boosting Self-Supervised Embeddings for Speech Enhancementcited by 24
- 2024MaskSR: Masked Language Model for Full-band Speech Restorationcited by 24
- 2024Universal Score-based Speech Enhancement with High Content Preservationcited by 23
- 2025AnyEnhance: A Unified Generative Model with Prompt-Guidance and Self-Critic for Voice Enhancementcited by 23
- 2023Miipher: A Robust Speech Restoration Model Integrating Self-Supervised Speech and Text Representationscited by 22
- 2025GenSE: Generative Speech Enhancement via Language Models using Hierarchical Modelingcited by 20
- 2025LLaSE-G1: Incentivizing Generalization Capability for LLaMA-based Speech Enhancementcited by 19
- 2024URGENT Challenge: Universality, Robustness, and Generalizability For Speech Enhancementcited by 17
- 2021DF-Conformer: Integrated architecture of Conv-TasNet and Conformer using linear complexity self-attention for speech enhancementcited by 16
- 2020Channel-wise Subband Input for Better Voice and Accompaniment Separation on High Resolution Musiccited by 13
- 2025Interspeech 2025 URGENT Speech Enhancement Challengecited by 12
- 2023Toward Universal Speech Enhancement for Diverse Input Conditionscited by 10
- 2022Audio Similarity is Unreliable as a Proxy for Audio Qualitycited by 7
- 2025Universal Speech Enhancement with Regression and Generative Mambacited by 7
50 most cited of 64 All 64 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025UniFlow: Unifying Speech Front-End Tasks via Continuous Generative Modeling
- 2025UniSE: A Unified Framework for Decoder-Only Autoregressive LM-Based Speech Enhancement
- 2026LIFT-SE: Linguistic Inference Followed by Flow Transformation for Generative Speech Enhancement ↗
- 2026StuPASE: Towards Low-Hallucination Studio-Quality Generative Speech Enhancement
- 2025A Composite Predictive-Generative Approach to Monaural Universal Speech Enhancement
- 2025DiTSE: High-Fidelity Generative Speech Enhancement via Latent Diffusion Transformers
- 2025Model as Loss: A Self-Consistent Training Paradigm
- 2025FlowSE: Efficient and High-Quality Speech Enhancement via Flow Matching