# mdctGAN: Taming transformer-based GAN for speech super-resolution with Modified DCT spectra

arXiv: https://arxiv.org/abs/2305.11104 · HTML: https://mlsanity.com/paper/2305.11104/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 22 references that are in the map.
- First posted on arXiv 18 May 2023.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Mixed Precision Training](https://mlsanity.com/paper/1710.03740/) (2017)
- [HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis](https://mlsanity.com/paper/2010.05646/) (2020)
- [MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis](https://mlsanity.com/paper/1910.06711/) (2019)
- [DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement](https://mlsanity.com/paper/2008.00264/) (2020)
- [Hi-Fi Multi-Speaker English TTS Dataset](https://mlsanity.com/paper/2104.01497/) (2021)
- [NU-Wave: A Diffusion Probabilistic Model for Neural Audio Upsampling](https://mlsanity.com/paper/2104.02321/) (2021)
- [Neural Vocoder is All You Need for Speech Super-resolution](https://mlsanity.com/paper/2203.14941/) (2022)
- [NU-Wave 2: A General Neural Audio Upsampling Model for Various Sampling Rates](https://mlsanity.com/paper/2206.08545/) (2022)
- [Audio Super Resolution using Neural Networks](https://mlsanity.com/paper/1708.00853/) (2017)
- [DCCRN+: Channel-wise Subband DCCRN with SNR Estimation for Speech Enhancement](https://mlsanity.com/paper/2106.08672/) (2021)
- [Conditioning and Sampling in Variational Diffusion Models for Speech Super-Resolution](https://mlsanity.com/paper/2210.15793/) (2022)
- [WSRGlow: A Glow-based Waveform Generative Model for Audio Super-Resolution](https://mlsanity.com/paper/2106.08507/) (2021)
- [Self-Attention for Audio Super-Resolution](https://mlsanity.com/paper/2108.11637/) (2021)
- [Nonparallel High-Quality Audio Super Resolution with Domain Adaptation and Resampling CycleGANs](https://mlsanity.com/paper/2210.15887/) (2022)

## Built on by (most cited first, 6 of 6)

- [Audio Super-Resolution with Latent Bridge Models](https://mlsanity.com/paper/2509.17609/) (2025)
- [A2SB: Audio-to-Audio Schrodinger Bridges](https://mlsanity.com/paper/2501.11311/) (2025)
- [Vibravox: A Dataset of French Speech Captured with Body-conduction Audio Sensors](https://mlsanity.com/paper/2407.11828/) (2024)
- [CodecFlow: Efficient Bandwidth Extension via Conditional Flow Matching in Neural Codec Latent Space](https://mlsanity.com/paper/2603.02022/) (2026)
- [LatentFlowSR: High-Fidelity Audio Super-Resolution via Noise-Robust Latent Flow Matching](https://mlsanity.com/paper/2604.09188/) (2026)
- [CogSR: Semantic-Aware Speech Super-Resolution via Chain-of-Thought Guided Flow Matching](https://mlsanity.com/paper/2512.16304/) (2025)

## Abstract (arXiv)

Speech super-resolution (SSR) aims to recover a high resolution (HR) speech from its corresponding low resolution (LR) counterpart. Recent SSR methods focus more on the reconstruction of the magnitude spectrogram, ignoring the importance of phase reconstruction, thereby limiting the recovery quality. To address this issue, we propose mdctGAN, a novel SSR framework based on modified discrete cosine transform (MDCT). By adversarial learning in the MDCT domain, our method reconstructs HR speeches in a phase-aware manner without vocoders or additional post-processing. Furthermore, by learning frequency consistent features with self-attentive mechanism, mdctGAN guarantees a high quality speech reconstruction. For VCTK corpus dataset, the experiment results show that our model produces natural auditory quality with high MOS and PESQ scores. It also achieves the state-of-the-art log-spectral-distance (LSD) performance on 48 kHz target resolution from various input rates. Code is available from https://github.com/neoncloud/mdctGAN