mlsanityData to 14 Oct 2026

UniPASE: A Generative Model for Universal Speech Enhancement with High Fidelity and Low Hallucinations

arXiv 2604.14606 ↗First posted 16 Apr 2026eess.AS cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Universal speech enhancement (USE) aims to restore speech signals from diverse distortions across multiple sampling rates. We propose UniPASE, an extension of the low-hallucination PASE framework tailored for USE. At its core is DeWavLM-Omni, a unified representation-level enhancement module fine-tuned from WavLM via knowledge distillation on a large-scale supervised multi-distortion dataset. This module directly converts degraded waveforms into clean and linguistically faithful phonetic representations, ensuring robust enhancement with minimal linguistic hallucination. Based on these enhanced phonetic representations, an Adapter generates enhanced acoustic representations containing rich acoustic details, which a neural Vocoder uses to reconstruct corresponding high-fidelity 16-kHz waveforms. A PostNet then converts the waveforms to 48~kHz before resampling them to their original rates, enabling seamless handling of inputs and outputs at multiple sampling rates. Experimental results on several evaluation datasets, covering sub-tasks and full tasks, demonstrate that UniPASE achieves superior or competitive performance compared with existing state-of-the-art models. The proposed model also serves as the backbone of our submission to the URGENT 2026 Challenge, which achieved 1st place in the objective evaluation. The source code and audio demos are available at https://github.com/xiaobin-rong/unipase/.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 64 of its 81 references that are in the map. First posted on arXiv 16 Apr 2026.

Builds on

64 of 81 references.

References · 64

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  2. 2022A ConvNet for the 2020scited by 3,579
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  5. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  6. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  7. 2016Least Squares Generative Adversarial Networkscited by 1,001
  8. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  9. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 476
  10. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 431
  11. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 390
  12. 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 347
38 more
  1. 2020FSD50K: An Open Dataset of Human-Labeled Sound Eventscited by 342
  2. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 231
  3. 2016FMA: A Dataset For Music Analysiscited by 207
  4. 2021NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasetscited by 192
  5. 2021DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressorscited by 172
  6. 2022Comparative layer-wise analysis of self-supervised speech modelscited by 158
  7. 2019WHAM!: Extending Speech Separation to Noisy Environmentscited by 156
  8. 2023Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesiscited by 153
  9. 2020The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Resultscited by 133
  10. 2024WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modelingcited by 129
  11. 2022StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberationcited by 105
  12. 2021Simple and Effective Zero-shot Cross-lingual Phoneme Recognitioncited by 79
  13. 2022Universal Speech Enhancement with Score-based Diffusioncited by 75
  14. 2024OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformercited by 74
  15. 2024Self-Supervised Speech Representations are More Phonetic than Semanticcited by 71
  16. 2024mHuBERT-147: A Compact Multilingual HuBERT Modelcited by 66
  17. 2023ICASSP 2023 Deep Noise Suppression Challengecited by 65
  18. 2024EARS: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberationcited by 64
  19. 2022TF-GridNet: Integrating Full- and Sub-Band Modeling for Speech Separationcited by 63
  20. 2022Pushing the limits of raw waveform speaker recognitioncited by 51
  21. 2024SpeechBERTScore: Reference-Aware Automatic Evaluation of Speech Generation Leveraging NLP Evaluation Metricscited by 38
  22. 2022VoiceFixer: A Unified Framework for High-Fidelity Speech Restorationcited by 37
  23. 2023SELM: Speech Enhancement Using Discrete Tokens and Language Modelscited by 36
  24. 2022Investigating self-supervised learning for speech enhancement and separationcited by 35
  25. 2022Boosting Self-Supervised Embeddings for Speech Enhancementcited by 24
  26. 2024MaskSR: Masked Language Model for Full-band Speech Restorationcited by 24
  27. 2024Universal Score-based Speech Enhancement with High Content Preservationcited by 23
  28. 2025AnyEnhance: A Unified Generative Model with Prompt-Guidance and Self-Critic for Voice Enhancementcited by 23
  29. 2023Miipher: A Robust Speech Restoration Model Integrating Self-Supervised Speech and Text Representationscited by 22
  30. 2025GenSE: Generative Speech Enhancement via Language Models using Hierarchical Modelingcited by 20
  31. 2025LLaSE-G1: Incentivizing Generalization Capability for LLaMA-based Speech Enhancementcited by 19
  32. 2024URGENT Challenge: Universality, Robustness, and Generalizability For Speech Enhancementcited by 17
  33. 2021DF-Conformer: Integrated architecture of Conv-TasNet and Conformer using linear complexity self-attention for speech enhancementcited by 16
  34. 2020Channel-wise Subband Input for Better Voice and Accompaniment Separation on High Resolution Musiccited by 13
  35. 2025Interspeech 2025 URGENT Speech Enhancement Challengecited by 12
  36. 2023Toward Universal Speech Enhancement for Diverse Input Conditionscited by 10
  37. 2022Audio Similarity is Unreliable as a Proxy for Audio Qualitycited by 7
  38. 2025Universal Speech Enhancement with Regression and Generative Mambacited by 7

50 most cited of 64 All 64 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025UniFlow: Unifying Speech Front-End Tasks via Continuous Generative Modeling
  2. 2025UniSE: A Unified Framework for Decoder-Only Autoregressive LM-Based Speech Enhancement
  3. 2026LIFT-SE: Linguistic Inference Followed by Flow Transformation for Generative Speech Enhancement ↗
  4. 2026StuPASE: Towards Low-Hallucination Studio-Quality Generative Speech Enhancement
  5. 2025A Composite Predictive-Generative Approach to Monaural Universal Speech Enhancement
  6. 2025DiTSE: High-Fidelity Generative Speech Enhancement via Latent Diffusion Transformers
  7. 2025Model as Loss: A Self-Consistent Training Paradigm
  8. 2025FlowSE: Efficient and High-Quality Speech Enhancement via Flow Matching