# HiFi++: a Unified Framework for Bandwidth Extension and Speech Enhancement

arXiv: https://arxiv.org/abs/2203.13086 · HTML: https://mlsanity.com/paper/2203.13086/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 60 references that are in the map.
- First posted on arXiv 24 Mar 2022.

## Builds on (most cited first, 30 of 33)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [Autoencoding beyond pixels using a learned similarity metric](https://mlsanity.com/paper/1512.09300/) (2015)
- [MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis](https://mlsanity.com/paper/1910.06711/) (2019)
- [WaveGlow: A Flow-based Generative Network for Speech Synthesis](https://mlsanity.com/paper/1811.00002/) (2018)
- [DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressors](https://mlsanity.com/paper/2010.15258/) (2020)
- [Real Time Speech Enhancement in the Waveform Domain](https://mlsanity.com/paper/2006.12847/) (2020)
- [DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors](https://mlsanity.com/paper/2110.01763/) (2021)
- [MOSNet: Deep Learning based Objective Assessment for Voice Conversion](https://mlsanity.com/paper/1904.08352/) (2019)
- [Wave-U-Net: A Multi-Scale Neural Network for End-to-End Audio Source Separation](https://mlsanity.com/paper/1806.03185/) (2018)
- [MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement](https://mlsanity.com/paper/2104.03538/) (2021)
- [MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement](https://mlsanity.com/paper/1905.04874/) (2019)
- [Music Source Separation in the Waveform Domain](https://mlsanity.com/paper/1911.13254/) (2019)
- [The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methods](https://mlsanity.com/paper/1804.04262/) (2018)
- [Generative Multi-Adversarial Networks](https://mlsanity.com/paper/1611.01673/) (2016)
- [VoiceFixer: Toward General Speech Restoration with Neural Vocoder](https://mlsanity.com/paper/2109.13731/) (2021)
- [Decoupling Magnitude and Phase Estimation with Deep ResUNet for Music Source Separation](https://mlsanity.com/paper/2109.05418/) (2021)
- [SEANet: A Multi-modal Speech Enhancement Network](https://mlsanity.com/paper/2009.02095/) (2020)
- [Temporal FiLM: Capturing Long-Range Sequence Dependencies with Feature-Wise Modulations](https://mlsanity.com/paper/1909.06628/) (2019)
- [Differentiable Consistency Constraints for Improved Deep Speech Enhancement](https://mlsanity.com/paper/1811.08521/) (2018)
- [Audio Super Resolution using Neural Networks](https://mlsanity.com/paper/1708.00853/) (2017)
- [Real-time Speech Frequency Bandwidth Extension](https://mlsanity.com/paper/2010.10677/) (2020)
- [Dual-branch Attention-In-Attention Transformer for single-channel speech enhancement](https://mlsanity.com/paper/2110.06467/) (2021)
- [GAN Vocoder: Multi-Resolution Discriminator Is All You Need](https://mlsanity.com/paper/2103.05236/) (2021)

## Built on by (most cited first, 30 of 49)

- [Speech Enhancement and Dereverberation with Diffusion-based Generative Models](https://mlsanity.com/paper/2208.05830/) (2022)
- [StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberation](https://mlsanity.com/paper/2212.11851/) (2022)
- [Analysing Diffusion-based Generative Approaches versus Discriminative Approaches for Speech Restoration](https://mlsanity.com/paper/2211.02397/) (2022)
- [EmergentTTS-Eval: Evaluating TTS Models on Complex Prosodic, Expressiveness, and Linguistic Challenges Using Model-as-a-Judge](https://mlsanity.com/paper/2505.23009/) (2025)
- [Blind Audio Bandwidth Extension: A Diffusion-Based Zero-Shot Approach](https://mlsanity.com/paper/2306.01433/) (2023)
- [FlowSE: Flow Matching-based Speech Enhancement](https://mlsanity.com/paper/2508.06840/) (2025)
- [InSerter: Speech Instruction Following with Unsupervised Interleaved Pre-training](https://mlsanity.com/paper/2503.02769/) (2025)
- [Diffusion Posterior Sampling for Informed Single-Channel Dereverberation](https://mlsanity.com/paper/2306.12286/) (2023)
- [Meta Learning Text-to-Speech Synthesis in over 7000 Languages](https://mlsanity.com/paper/2406.06403/) (2024)
- [1000 African Voices: Advancing inclusive multi-speaker multi-accent speech synthesis](https://mlsanity.com/paper/2406.11727/) (2024)
- [UnDiff: Unsupervised Voice Restoration with Unconditional Diffusion Model](https://mlsanity.com/paper/2306.00721/) (2023)
- [FINALLY: fast and universal speech enhancement with studio-like quality](https://mlsanity.com/paper/2410.05920/) (2024)
- [Discrete Diffusion for Generative Modeling of Text-Aligned Speech Tokens](https://mlsanity.com/paper/2509.20060/) (2025)
- [On the Behavior of Intrusive and Non-intrusive Speech Enhancement Metrics in Predictive and Generative Settings](https://mlsanity.com/paper/2306.03014/) (2023)
- [Diffusion Buffer: Online Diffusion-based Speech Enhancement with Sub-Second Latency](https://mlsanity.com/paper/2506.02908/) (2025)
- [Iterative autoregression: a novel trick to improve your low-latency speech enhancement model](https://mlsanity.com/paper/2211.01751/) (2022)
- [Vibravox: A Dataset of French Speech Captured with Body-conduction Audio Sensors](https://mlsanity.com/paper/2407.11828/) (2024)
- [Anonymising Elderly and Pathological Speech: Voice Conversion Using DDSP and Query-by-Example](https://mlsanity.com/paper/2410.15500/) (2024)
- [Children's Voice Privacy: First Steps And Emerging Challenges](https://mlsanity.com/paper/2506.00100/) (2025)
- [A Composite Predictive-Generative Approach to Monaural Universal Speech Enhancement](https://mlsanity.com/paper/2505.24576/) (2025)
- [Robust One-step Speech Enhancement via Consistency Distillation](https://mlsanity.com/paper/2507.05688/) (2025)
- [VoiceBridge: General Speech Restoration with One-step Latent Bridge Models](https://mlsanity.com/paper/2509.25275/) (2025)
- [Real-Time Streamable Generative Speech Restoration with Flow Matching](https://mlsanity.com/paper/2512.19442/) (2025)
- [In-the-wild Speech Emotion Conversion Using Disentangled Self-Supervised Representations and Neural Vocoder-based Resynthesis](https://mlsanity.com/paper/2306.01916/) (2023)
- [Wind Noise Reduction with a Diffusion-based Stochastic Regeneration Model](https://mlsanity.com/paper/2306.12867/) (2023)
- [Low-latency Real-time Voice Conversion on CPU](https://mlsanity.com/paper/2311.00873/) (2023)
- [Conditional Latent Diffusion-Based Speech Enhancement Via Dual Context Learning](https://mlsanity.com/paper/2501.10052/) (2025)
- [Speech Enhancement Based on Drifting Models](https://mlsanity.com/paper/2604.24199/) (2026)
- [The Limits of Reference-Free Speech Quality Metrics as Evaluators and Rewards on Modern Text-to-Speech](https://mlsanity.com/paper/2609.13150/) (2026)
- [Speech Boosting: Low-Latency Live Speech Enhancement for TWS Earbuds](https://mlsanity.com/paper/2409.18705/) (2024)

## Abstract (arXiv)

Generative adversarial networks have recently demonstrated outstanding performance in neural vocoding outperforming best autoregressive and flow-based models. In this paper, we show that this success can be extended to other tasks of conditional audio generation. In particular, building upon HiFi vocoders, we propose a novel HiFi++ general framework for bandwidth extension and speech enhancement. We show that with the improved generator architecture, HiFi++ performs better or comparably with the state-of-the-art in these tasks while spending significantly less computational resources. The effectiveness of our approach is validated through a series of extensive experiments.