# NU-Wave: A Diffusion Probabilistic Model for Neural Audio Upsampling

arXiv: https://arxiv.org/abs/2104.02321 · HTML: https://mlsanity.com/paper/2104.02321/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 22 of its 32 references that are in the map.
- First posted on arXiv 6 Apr 2021.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [WaveGrad: Estimating Gradients for Waveform Generation](https://mlsanity.com/paper/2009.00713/) (2020)
- [PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models](https://mlsanity.com/paper/2003.03808/) (2020)
- [WaveGlow: A Flow-based Generative Network for Speech Synthesis](https://mlsanity.com/paper/1811.00002/) (2018)
- [Efficient Neural Audio Synthesis](https://mlsanity.com/paper/1802.08435/) (2018)
- [Adversarial score matching and improved sampling for image generation](https://mlsanity.com/paper/2009.05475/) (2020)
- [WaveFlow: A Compact Flow-based Model for Raw Audio](https://mlsanity.com/paper/1912.01219/) (2019)
- [Temporal FiLM: Capturing Long-Range Sequence Dependencies with Feature-Wise Modulations](https://mlsanity.com/paper/1909.06628/) (2019)
- [Audio Super Resolution using Neural Networks](https://mlsanity.com/paper/1708.00853/) (2017)
- [Bandwidth Extension on Raw Audio via Generative Adversarial Networks](https://mlsanity.com/paper/1903.09027/) (2019)

## Built on by (most cited first, 30 of 50)

- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [A Survey on Generative Diffusion Model](https://mlsanity.com/paper/2209.02646/) (2022)
- [On Fast Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2106.00132/) (2021)
- [TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation](https://mlsanity.com/paper/2303.04248/) (2023)
- [Diffusion Models for Reinforcement Learning: A Survey](https://mlsanity.com/paper/2311.01223/) (2023)
- [PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Dependent Adaptive Prior](https://mlsanity.com/paper/2106.06406/) (2021)
- [Universal Speech Enhancement with Score-based Diffusion](https://mlsanity.com/paper/2206.03065/) (2022)
- [A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AI](https://mlsanity.com/paper/2303.13336/) (2023)
- [Multi-Source Diffusion Models for Simultaneous Music Generation and Separation](https://mlsanity.com/paper/2302.02257/) (2023)
- [Discrete Contrastive Diffusion for Cross-Modal Music and Image Generation](https://mlsanity.com/paper/2206.07771/) (2022)
- [Multi-instrument Music Synthesis with Spectrogram Diffusion](https://mlsanity.com/paper/2206.05408/) (2022)
- [AudioSR: Versatile Audio Super-resolution at Scale](https://mlsanity.com/paper/2309.07314/) (2023)
- [Neural Vocoder is All You Need for Speech Super-resolution](https://mlsanity.com/paper/2203.14941/) (2022)
- [NU-Wave 2: A General Neural Audio Upsampling Model for Various Sampling Rates](https://mlsanity.com/paper/2206.08545/) (2022)
- [Analysing Diffusion-based Generative Approaches versus Discriminative Approaches for Speech Restoration](https://mlsanity.com/paper/2211.02397/) (2022)
- [Boundary Guided Learning-Free Semantic Control with Diffusion Models](https://mlsanity.com/paper/2302.08357/) (2023)
- [Vision+X: A Survey on Multimodal Learning in the Light of Data](https://mlsanity.com/paper/2210.02884/) (2022)
- [Physics-Driven Diffusion Models for Impact Sound Synthesis from Videos](https://mlsanity.com/paper/2303.16897/) (2023)
- [Conditioning and Sampling in Variational Diffusion Models for Speech Super-Resolution](https://mlsanity.com/paper/2210.15793/) (2022)
- [A Survey on Diffusion Models for Recommender Systems](https://mlsanity.com/paper/2409.05033/) (2024)
- [FLowHigh: Towards Efficient and High-Quality Audio Super-Resolution with Single-Step Flow Matching](https://mlsanity.com/paper/2501.04926/) (2025)
- [Audio Super-Resolution with Latent Bridge Models](https://mlsanity.com/paper/2509.17609/) (2025)
- [High-Quality Visually-Guided Sound Separation from Diverse Categories](https://mlsanity.com/paper/2308.00122/) (2023)
- [A2SB: Audio-to-Audio Schrodinger Bridges](https://mlsanity.com/paper/2501.11311/) (2025)
- [DiffSDS: A language diffusion model for protein backbone inpainting under geometric conditions and constraints](https://mlsanity.com/paper/2301.09642/) (2023)
- [Wave-U-Mamba: An End-To-End Framework For High-Quality And Efficient Speech Super Resolution](https://mlsanity.com/paper/2409.09337/) (2024)
- [TUNet: A Block-online Bandwidth Extension Model based on Transformers and Self-supervised Pretraining](https://mlsanity.com/paper/2110.13492/) (2021)
- [PhaseAug: A Differentiable Augmentation for Speech Synthesis to Simulate One-to-Many Mapping](https://mlsanity.com/paper/2211.04610/) (2022)
- [mdctGAN: Taming transformer-based GAN for speech super-resolution with Modified DCT spectra](https://mlsanity.com/paper/2305.11104/) (2023)
- [AIGC for Industrial Time Series: From Deep Generative Models to Large Generative Models](https://mlsanity.com/paper/2407.11480/) (2024)

## Abstract (arXiv)

In this work, we introduce NU-Wave, the first neural audio upsampling model to produce waveforms of sampling rate 48kHz from coarse 16kHz or 24kHz inputs, while prior works could generate only up to 16kHz. NU-Wave is the first diffusion probabilistic model for audio super-resolution which is engineered based on neural vocoders. NU-Wave generates high-quality audio that achieves high performance in terms of signal-to-noise ratio (SNR), log-spectral distance (LSD), and accuracy of the ABX test. In all cases, NU-Wave outperforms the baseline models despite the substantially smaller model capacity (3.0M parameters) than baselines (5.4-21%). The audio samples of our model are available at https://mindslab-ai.github.io/nuwave, and the code will be made available soon.