# NU-Wave 2: A General Neural Audio Upsampling Model for Various Sampling Rates

arXiv: https://arxiv.org/abs/2206.08545 · HTML: https://mlsanity.com/paper/2206.08545/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 30 references that are in the map.
- First posted on arXiv 17 Jun 2022.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [Resolution-robust Large Mask Inpainting with Fourier Convolutions](https://mlsanity.com/paper/2109.07161/) (2021)
- [WaveGrad: Estimating Gradients for Waveform Generation](https://mlsanity.com/paper/2009.00713/) (2020)
- [Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform](https://mlsanity.com/paper/1804.02815/) (2018)
- [NU-Wave: A Diffusion Probabilistic Model for Neural Audio Upsampling](https://mlsanity.com/paper/2104.02321/) (2021)
- [VoiceFixer: Toward General Speech Restoration with Neural Vocoder](https://mlsanity.com/paper/2109.13731/) (2021)
- [Temporal FiLM: Capturing Long-Range Sequence Dependencies with Feature-Wise Modulations](https://mlsanity.com/paper/1909.06628/) (2019)
- [Audio Super Resolution using Neural Networks](https://mlsanity.com/paper/1708.00853/) (2017)
- [WSRGlow: A Glow-based Waveform Generative Model for Audio Super-Resolution](https://mlsanity.com/paper/2106.08507/) (2021)
- [Bandwidth Extension on Raw Audio via Generative Adversarial Networks](https://mlsanity.com/paper/1903.09027/) (2019)
- [TUNet: A Block-online Bandwidth Extension Model based on Transformers and Self-supervised Pretraining](https://mlsanity.com/paper/2110.13492/) (2021)

## Built on by (most cited first, 30 of 36)

- [TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation](https://mlsanity.com/paper/2303.04248/) (2023)
- [StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberation](https://mlsanity.com/paper/2212.11851/) (2022)
- [A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AI](https://mlsanity.com/paper/2303.13336/) (2023)
- [Solving Audio Inverse Problems with a Diffusion Model](https://mlsanity.com/paper/2210.15228/) (2022)
- [Diffusion-based Generative Speech Source Separation](https://mlsanity.com/paper/2210.17327/) (2022)
- [AudioSR: Versatile Audio Super-resolution at Scale](https://mlsanity.com/paper/2309.07314/) (2023)
- [DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust Voice Conversion](https://mlsanity.com/paper/2305.15816/) (2023)
- [HierSpeech++: Bridging the Gap between Semantic and Acoustic Representation of Speech by Hierarchical Variational Inference for Zero-shot Speech Synthesis](https://mlsanity.com/paper/2311.12454/) (2023)
- [Analysing Diffusion-based Generative Approaches versus Discriminative Approaches for Speech Restoration](https://mlsanity.com/paper/2211.02397/) (2022)
- [Blind Audio Bandwidth Extension: A Diffusion-Based Zero-Shot Approach](https://mlsanity.com/paper/2306.01433/) (2023)
- [Conditioning and Sampling in Variational Diffusion Models for Speech Super-Resolution](https://mlsanity.com/paper/2210.15793/) (2022)
- [Avocodo: Generative Adversarial Network for Artifact-free Vocoder](https://mlsanity.com/paper/2206.13404/) (2022)
- [FLowHigh: Towards Efficient and High-Quality Audio Super-Resolution with Single-Step Flow Matching](https://mlsanity.com/paper/2501.04926/) (2025)
- [AERO: Audio Super Resolution in the Spectral Domain](https://mlsanity.com/paper/2211.12232/) (2022)
- [Audio Super-Resolution with Latent Bridge Models](https://mlsanity.com/paper/2509.17609/) (2025)
- [PeriodWave: Multi-Period Flow Matching for High-Fidelity Waveform Generation](https://mlsanity.com/paper/2408.07547/) (2024)
- [Wave-U-Mamba: An End-To-End Framework For High-Quality And Efficient Speech Super Resolution](https://mlsanity.com/paper/2409.09337/) (2024)
- [PhaseAug: A Differentiable Augmentation for Speech Synthesis to Simulate One-to-Many Mapping](https://mlsanity.com/paper/2211.04610/) (2022)
- [mdctGAN: Taming transformer-based GAN for speech super-resolution with Modified DCT spectra](https://mlsanity.com/paper/2305.11104/) (2023)
- [Vibravox: A Dataset of French Speech Captured with Body-conduction Audio Sensors](https://mlsanity.com/paper/2407.11828/) (2024)
- [Improving Diffusion Models's Data-Corruption Resistance using Scheduled Pseudo-Huber Loss](https://mlsanity.com/paper/2403.16728/) (2024)
- [Vector Quantized Diffusion Model Based Speech Bandwidth Extension](https://mlsanity.com/paper/2409.05784/) (2024)
- [VoiceBridge: General Speech Restoration with One-step Latent Bridge Models](https://mlsanity.com/paper/2509.25275/) (2025)
- [Inference-time Scaling for Diffusion-based Audio Super-resolution](https://mlsanity.com/paper/2508.02391/) (2025)
- [UniverSR: Unified and Versatile Audio Super-Resolution via Vocoder-Free Flow Matching](https://mlsanity.com/paper/2510.00771/) (2025)
- [Nonparallel High-Quality Audio Super Resolution with Domain Adaptation and Resampling CycleGANs](https://mlsanity.com/paper/2210.15887/) (2022)
- [A High-Fidelity Speech Super Resolution Network using a Complex Global Attention Module with Spectro-Temporal Loss](https://mlsanity.com/paper/2507.00229/) (2025)
- [CodecFlow: Efficient Bandwidth Extension via Conditional Flow Matching in Neural Codec Latent Space](https://mlsanity.com/paper/2603.02022/) (2026)
- [LatentFlowSR: High-Fidelity Audio Super-Resolution via Noise-Robust Latent Flow Matching](https://mlsanity.com/paper/2604.09188/) (2026)
- [Edge Storage Management Recipe with Zero-Shot Data Compression for Road Anomaly Detection](https://mlsanity.com/paper/2307.04298/) (2023)

## Abstract (arXiv)

Conventionally, audio super-resolution models fixed the initial and the target sampling rates, which necessitate the model to be trained for each pair of sampling rates. We introduce NU-Wave 2, a diffusion model for neural audio upsampling that enables the generation of 48 kHz audio signals from inputs of various sampling rates with a single model. Based on the architecture of NU-Wave, NU-Wave 2 uses short-time Fourier convolution (STFC) to generate harmonics to resolve the main failure modes of NU-Wave, and incorporates bandwidth spectral feature transform (BSFT) to condition the bandwidths of inputs in the frequency domain. We experimentally demonstrate that NU-Wave 2 produces high-resolution audio regardless of the sampling rate of input while requiring fewer parameters than other models. The official code and the audio samples are available at https://mindslab-ai.github.io/nuwave2.