mlsanityData to 7 Oct 2026

Music Enhancement via Image Translation and Vocoding

arXiv 2204.13289 ↗First posted 28 Apr 2022cs.SD cs.LG eess.AScited by 11

Not on the map. Why?

Abstract (arXiv)

Consumer-grade music recordings such as those captured by mobile devices typically contain distortions in the form of background noise, reverb, and microphone-induced EQ. This paper presents a deep learning approach to enhance low-quality music recordings by combining (i) an image-to-image translation model for manipulating audio in its mel-spectrogram representation and (ii) a music vocoding model for mapping synthetically generated mel-spectrograms to perceptually realistic waveforms. We find that this approach to music enhancement outperforms baselines which use classical methods for mel-spectrogram inversion and an end-to-end approach directly mapping noisy waveforms to clean waveforms. Additionally, in evaluating the proposed method with a listening test, we analyze the reliability of common audio enhancement evaluation metrics when used in the music domain.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 21 of its 32 references that are in the map. First posted on arXiv 28 Apr 2022.

Builds on

21 of 32 references.

References · 21

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  4. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  5. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
  6. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  7. 2018Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separationcited by 400
  8. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
  9. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
  10. 2020DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressorscited by 235
  11. 2018Fr'echet Audio Distance: A Metric for Evaluating Music Enhancement Algorithmscited by 221
  12. 2020Real Time Speech Enhancement in the Waveform Domaincited by 183
9 more
  1. 2019Music Source Separation in the Waveform Domaincited by 98
  2. 2021TorchAudio: Building Blocks for Audio and Speech Processingcited by 83
  3. 2021Interspeech 2021 Deep Noise Suppression Challengecited by 43
  4. 2021CDPAM: Contrastive learning for perceptual audio similaritycited by 42
  5. 2020HiFi-GAN: High-Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networkscited by 40
  6. 2021GAN Vocoder: Multi-Resolution Discriminator Is All You Needcited by 16
  7. 2016Deep convolutional networks on the pitch spiral for musical instrument recognitioncited by 14
  8. 2021High Fidelity Speech Regeneration with Application to Speech Enhancementcited by 10
  9. 2019Towards Generalized Speech Enhancement with Generative Adversarial Networkscited by 8

Built on by

Cited by year

Citing papers per year: 2022: 5; 2023: 3; 2024: 2; 2025: 1.

11 papers.

  1. 2023GibbsDDRM: A Partially Collapsed Gibbs Sampler for Solving Blind Inverse Problems with Denoising Diffusion Restorationcited by 58
  2. 2022Multi-instrument Music Synthesis with Spectrogram Diffusioncited by 42
  3. 2024MaskSR: Masked Language Model for Full-band Speech Restorationcited by 24
  4. 2022Unsupervised vocal dereverberation with diffusion-based generative modelscited by 18
  5. 2024Unsupervised Blind Joint Dereverberation and Room Acoustics Estimation with Diffusion Modelscited by 11
  6. 2022Music Separation Enhancement with Generative Modelingcited by 8
  7. 2025SonicMaster: Towards Controllable All-in-One Music Restoration and Masteringcited by 6
  8. 2022Mel Spectrogram Inversion with Stable Pitchcited by 2
  9. 2023Exploiting Time-Frequency Conformers for Music Audio Enhancementcited by 2
  10. 2022Audio Denoising for Robust Audio Fingerprintingcited by 1
  11. 2023Music Augmentation and Denoising For Peak-Based Audio Fingerprintingcited by 0