mlsanityData to 14 Oct 2026

EDMSound: Spectrogram Based Diffusion Models for Efficient and High-Quality Audio Synthesis

arXiv 2311.08667 ↗First posted 15 Nov 2023cs.SD eess.AScited by 15

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Audio diffusion models can synthesize a wide variety of sounds. Existing models often operate on the latent domain with cascaded phase recovery modules to reconstruct waveform. This poses challenges when generating high-fidelity audio. In this paper, we propose EDMSound, a diffusion-based generative model in spectrogram domain under the framework of elucidated diffusion models (EDM). Combining with efficient deterministic sampler, we achieved similar Fr'echet audio distance (FAD) score as top-ranked baseline with only 10 steps and reached state-of-the-art performance with 50 steps on the DCASE2023 foley sound generation benchmark. We also revealed a potential concern regarding diffusion based audio generation models that they tend to generate samples with high perceptual similarity to the data from training data. Project page: https://agentcooper2002.github.io/EDMSound/

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 38 of its 44 references that are in the map. First posted on arXiv 15 Nov 2023.

Builds on

38 of 44 references.

Closest ancestors · 11

Same direction, newest first

  1. 2023Latent Diffusion Model Based Foley Sound Generation System For DCASE Challenge 2023 Task 725 May
  2. 2023Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model24 Apr
  3. 2023AUDIT: Audio Editing by Following Instructions with Latent Diffusion Models3 Apr
  4. 2023Noise2Music: Text-conditioned Music Generation with Diffusion Models8 Feb
  5. 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models30 Jan
  6. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Models29 Jan
  7. 2022GAN You Hear Me? Reclaiming Unconditional Speech Synthesis from Diffusion Models11 Oct
  8. 2022Speech Enhancement and Dereverberation with Diffusion-based Generative Models11 Aug
  9. 2022Diffsound: Discrete Diffusion Model for Text-to-sound Generation20 Jul
  10. 2022Multi-instrument Music Synthesis with Spectrogram Diffusion11 Jun
  11. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesis21 Sep

Other references · 27

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2020Denoising Diffusion Implicit Modelscited by 9,781
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  6. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  7. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
  8. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,979
  9. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  10. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 938
  11. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 752
  12. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 719
15 more
  1. 2019PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognitioncited by 574
  2. 2022Fast Sampling of Diffusion Models with Exponential Integratorcited by 532
  3. 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Modelscited by 450
  4. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 424
  5. 2022Masked Autoencoders that Listencited by 273
  6. 2018Adversarial Audio Synthesiscited by 225
  7. 2018Fr'echet Audio Distance: A Metric for Evaluating Music Enhancement Algorithmscited by 225
  8. 2023Understanding and Mitigating Copying in Diffusion Modelscited by 209
  9. 2022It's Raw! Audio Generation with State-Space Modelscited by 171
  10. 2016SampleRNN: An Unconditional End-to-End Neural Audio Generation Modelcited by 110
  11. 2019Low Bit-Rate Speech Coding with VQ-VAE and a WaveNet Decodercited by 49
  12. 2022Full-band General Audio Synthesis with Score-based Diffusioncited by 32
  13. 2023Foley Sound Synthesis at the DCASE 2023 Challengecited by 19
  14. 2022Simple Pooling Front-ends For Efficient Audio Classificationcited by 9
  15. 2023A Demand-Driven Perspective on Generative Audio AIcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 6; 2026: 3.

15 papers.

Sort:

By area:

  1. 2024Diff-A-Riff: Musical Accompaniment Co-creation via Latent Diffusion Modelscited by 33
  2. 2024Music2Latent: Consistency Autoencoders for Latent Audio Compressioncited by 33
  3. 2024Presto! Distilling Steps and Layers for Accelerating Music Generationcited by 16
  4. 2024ImmerseDiffusion: A Generative Spatial Audio Latent Diffusion Modelcited by 11
  5. 2024Multi-Track MusicLDM: Towards Versatile Music Generation with Latent Diffusion Modelcited by 5
  6. 2025Post-Training Quantization for Audio Diffusion Transformerscited by 3
  7. 2025User-guided Generative Source Separationcited by 2
  8. 2024IDEQ -- Improving Diffusion Models for the Traveling Salesman Problem (TSP) by Leveraging the Structure of the Solution Spacecited by 1
  9. 2025Audio Generation Through Score-Based Generative Modeling: Design Principles and Implementationcited by 1
  10. 2025Learning Linearity in Audio Consistency Autoencoders via Implicit Regularizationcited by 1
  11. 2026Covariance-aware sampling for Diffusion Modelscited by 1
  12. 2025HARP 2.0: Expanding Hosted, Asynchronous, Remote Processing for Deep Learning in the DAWcited by 0
3 more
  1. 2025MARS: Sound Generation via Multi-Channel Autoregression on Spectrogramscited by 0
  2. 2026Repurposing a Speech Classifier for Guided Diffusion-Based Speech Generationcited by 0
  3. 2026A Production-Oriented Framework for Evaluation of SFX Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Diff-VS: Efficient Audio-Aware Diffusion U-Net for Vocals Separation
  2. 2023ArchiSound: Audio Generation with Diffusion
  3. 2023From Discrete Tokens to High-Fidelity Audio Using Multi-Band Diffusion
  4. 2026AudioX-Turbo: A Unified Framework for Efficient Anything-to-Audio Generation
  5. 2025Audio Palette: A Diffusion Transformer with Multi-Signal Conditioning for Controllable Foley Synthesis
  6. 2023Controllable Music Production with Diffusion Models and Guidance Gradients
  7. 2026Exploring Efficient Waveform Diffusion Models for Foley Sound Generation
  8. 2025Score Distillation Sampling for Audio: Source Separation, Synthesis, and Beyond