# Nonparallel High-Quality Audio Super Resolution with Domain Adaptation and Resampling CycleGANs

arXiv: https://arxiv.org/abs/2210.15887 · HTML: https://mlsanity.com/paper/2210.15887/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 32 references that are in the map.
- First posted on arXiv 28 Oct 2022.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Language Modeling with Gated Convolutional Networks](https://mlsanity.com/paper/1612.08083/) (2016)
- [Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech](https://mlsanity.com/paper/2106.06103/) (2021)
- [Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks](https://mlsanity.com/paper/1602.07868/) (2016)
- [Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogram](https://mlsanity.com/paper/1910.11480/) (2019)
- [The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Results](https://mlsanity.com/paper/2005.13981/) (2020)
- [UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generation](https://mlsanity.com/paper/2106.07889/) (2021)
- [NU-Wave: A Diffusion Probabilistic Model for Neural Audio Upsampling](https://mlsanity.com/paper/2104.02321/) (2021)
- [Neural Vocoder is All You Need for Speech Super-resolution](https://mlsanity.com/paper/2203.14941/) (2022)
- [NU-Wave 2: A General Neural Audio Upsampling Model for Various Sampling Rates](https://mlsanity.com/paper/2206.08545/) (2022)
- [WSRGlow: A Glow-based Waveform Generative Model for Audio Super-Resolution](https://mlsanity.com/paper/2106.08507/) (2021)
- [GAN Vocoder: Multi-Resolution Discriminator Is All You Need](https://mlsanity.com/paper/2103.05236/) (2021)
- [NU-GAN: High resolution neural upsampling with GAN](https://mlsanity.com/paper/2010.11362/) (2020)
- [Bandwidth Extension on Raw Audio via Generative Adversarial Networks](https://mlsanity.com/paper/1903.09027/) (2019)
- [TUNet: A Block-online Bandwidth Extension Model based on Transformers and Self-supervised Pretraining](https://mlsanity.com/paper/2110.13492/) (2021)
- [WaveCycleGAN: Synthetic-to-natural speech waveform conversion using cycle-consistent adversarial networks](https://mlsanity.com/paper/1809.10288/) (2018)
- [WaveCycleGAN2: Time-domain Neural Post-filter for Speech Waveform Generation](https://mlsanity.com/paper/1904.02892/) (2019)
- [Joint domain adaptation and speech bandwidth extension using time-domain GANs for speaker verification](https://mlsanity.com/paper/2203.16614/) (2022)

## Built on by (most cited first, 2 of 2)

- [mdctGAN: Taming transformer-based GAN for speech super-resolution with Modified DCT spectra](https://mlsanity.com/paper/2305.11104/) (2023)
- [MSECG: Incorporating Mamba for Robust and Efficient ECG Super-Resolution](https://mlsanity.com/paper/2412.04861/) (2024)

## Abstract (arXiv)

Neural audio super-resolution models are typically trained on low- and high-resolution audio signal pairs. Although these methods achieve highly accurate super-resolution if the acoustic characteristics of the input data are similar to those of the training data, challenges remain: the models suffer from quality degradation for out-of-domain data, and paired data are required for training. To address these problems, we propose Dual-CycleGAN, a high-quality audio super-resolution method that can utilize unpaired data based on two connected cycle consistent generative adversarial networks (CycleGAN). Our method decomposes the super-resolution method into domain adaptation and resampling processes to handle acoustic mismatch in the unpaired low- and high-resolution signals. The two processes are then jointly optimized within the CycleGAN framework. Experimental results verify that the proposed method significantly outperforms conventional methods when paired data are not available. Code and audio samples are available from https://chomeyama.github.io/DualCycleGAN-Demo/.