mlsanityData to 7 Oct 2026

CRASH: Raw Audio Score-based Generative Modeling for Controllable High-resolution Drum Sound Synthesis

arXiv 2106.07431 ↗First posted 14 Jun 2021cs.SD cs.LG eess.AScited by 37

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

In this paper, we propose a novel score-base generative model for unconditional raw audio synthesis. Our proposal builds upon the latest developments on diffusion process modeling with stochastic differential equations, which already demonstrated promising results on image generation. We motivate novel heuristics for the choice of the diffusion processes better suited for audio generation, and consider the use of a conditional U-Net to approximate the score function. While previous approaches on diffusion models on audio were mainly designed as speech vocoders in medium resolution, our method termed CRASH (Controllable Raw Audio Synthesis with High-resolution) allows us to generate short percussive sounds in 44.1kHz in a controllable way. Through extensive experiments, we showcase on a drum sound generation task the numerous sampling schemes offered by our method (unconditional generation, deterministic generation, inpainting, interpolation, variations, class-conditional sampling) and propose the class-mixing sampling, a novel way to generate "hybrid" sounds. Our proposed method closes the gap with GAN-based methods on raw audio, while offering more flexible generation capabilities with lighter and easier-to-train models.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 28 of its 33 references that are in the map. First posted on arXiv 14 Jun 2021.

Builds on

28 of 33 references.

Closest ancestors · 1

Same direction, newest first

  1. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesis21 Sep

Other references · 27

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  7. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  8. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  9. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  10. 2014Conditional Generative Adversarial Netscited by 2,152
  11. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  12. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
15 more
  1. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  2. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 741
  3. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  4. 2020GANSpace: Discovering Interpretable GAN Controlscited by 612
  5. 2021Designing an Encoder for StyleGAN Image Manipulationcited by 542
  6. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 308
  7. 2018Adversarial Audio Synthesiscited by 222
  8. 2018Fr'echet Audio Distance: A Metric for Evaluating Music Enhancement Algorithmscited by 221
  9. 2019GANSynth: Adversarial Neural Audio Synthesiscited by 114
  10. 2016SampleRNN: An Unconditional End-to-End Neural Audio Generation Modelcited by 110
  11. 2020A Spectral Energy Distance for Parallel Speech Synthesiscited by 42
  12. 2019MelNet: A Generative Model for Audio in the Frequency Domaincited by 41
  13. 2020DrumGAN: Synthesis of Drum Sounds With Timbral Feature Conditioning Using Generative Adversarial Networkscited by 34
  14. 2019Conditioned-U-Net: Introducing a Control Mechanism in the U-Net for Multiple Source Separationscited by 27
  15. 2019Neural Drum Machine : An Interactive System for Real-time Synthesis of Drum Soundscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 10; 2023: 11; 2024: 11; 2025: 2; 2026: 3.

37 papers.

Sort:

By area:

  1. 2024Fast Timing-Conditioned Latent Audio Diffusioncited by 178
  2. 2022Protein structure generation via folding diffusioncited by 113
  3. 2024Long-form music generation with latent diffusioncited by 88
  4. 2023A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?cited by 84
  5. 2022Universal Speech Enhancement with Score-based Diffusioncited by 75
  6. 2023A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AIcited by 68
  7. 2023Multi-Source Diffusion Models for Simultaneous Music Generation and Separationcited by 63
  8. 2022Solving Audio Inverse Problems with a Diffusion Modelcited by 62
  9. 2023Enhancing Deep Reinforcement Learning: A Tutorial on Generative Diffusion Models in Network Optimizationcited by 56
  10. 2024Scaling Transformers for Low-Bitrate High-Quality Speech Codingcited by 48
  11. 2022Multi-instrument Music Synthesis with Spectrogram Diffusioncited by 42
  12. 2024Foundation Models for Music: A Surveycited by 34
25 more
  1. 2024Diff-A-Riff: Musical Accompaniment Co-creation via Latent Diffusion Modelscited by 33
  2. 2022Full-band General Audio Synthesis with Score-based Diffusioncited by 32
  3. 2023Controllable Music Production with Diffusion Models and Guidance Gradientscited by 29
  4. 2023The Ethical Implications of Generative Audio Models: A Systematic Literature Reviewcited by 18
  5. 2022Protein Language Models and Structure Prediction: Connection and Progressioncited by 17
  6. 2024T-FOLEY: A Controllable Waveform-Domain Diffusion Model for Temporal-Event-Guided Foley Sound Synthesiscited by 17
  7. 2022Evaluating generative audio systems and their metricscited by 16
  8. 2024Detecting Out-Of-Distribution Earth Observation Images with Diffusion Modelscited by 14
  9. 2024Improving Musical Accompaniment Co-creation via Diffusion Transformerscited by 11
  10. 2023UnDiff: Unsupervised Voice Restoration with Unconditional Diffusion Modelcited by 7
  11. 2023Timbre transfer using image-to-image denoising diffusion implicit modelscited by 7
  12. 2023Msanii: High Fidelity Music Synthesis on a Shoestring Budgetcited by 6
  13. 2023Zero-Shot Duet Singing Voices Separation with Diffusion Modelscited by 6
  14. 2022Differentiable Digital Signal Processing Mixture Model for Synthesis Parameter Extraction from Mixture of Harmonic Soundscited by 4
  15. 2022DrumGAN VST: A Plugin for Drum Sound Analysis/Synthesis With Autoencoding Generative Adversarial Networkscited by 4
  16. 2022Realistic Gramophone Noise Synthesis using a Diffusion Modelcited by 3
  17. 2024MambaFoley: Foley Sound Generation using Selective State-Space Modelscited by 3
  18. 2025The Inverse Drum Machine: Source Separation Through Joint Transcription and Analysis-by-Synthesiscited by 3
  19. 2024SYMPLEX: Controllable Symbolic Music Generation using Simplex Diffusion with Vocabulary Priorscited by 2
  20. 2023Distribution Preserving Source Separation With Time Frequency Predictive Modelscited by 1
  21. 2025Audio Generation Through Score-Based Generative Modeling: Design Principles and Implementationcited by 1
  22. 2026Drum Synthesis from Expressive Drum Grids via Neural Audio Codecscited by 1
  23. 2024Pictures Of MIDI: Controlled Music Generation via Graphical Prompts for Image-Based Diffusion Inpaintingcited by 0
  24. 2026Seconds-Aligned PCA-DAC Latent Diffusion for Symbolic-to-Audio Drum Renderingcited by 0
  25. 2026Differentiable Digital Signal Processing Mixture Model-Guided Diffusion for Synthesis Parameter Estimation from Harmonic Sound Mixturescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023EDMSound: Spectrogram Based Diffusion Models for Efficient and High-Quality Audio Synthesis
  2. 2025Noise-to-Notes: Diffusion-based Generation and Refinement for Automatic Drum Transcription
  3. 2025Score Distillation Sampling for Audio: Source Separation, Synthesis, and Beyond
  4. 2023From Discrete Tokens to High-Fidelity Audio Using Multi-Band Diffusion
  5. 2023AudioSR: Versatile Audio Super-resolution at Scale
  6. 2026AudioX-Turbo: A Unified Framework for Efficient Anything-to-Audio Generation
  7. 2024Conditional GAN for Enhancing Diffusion Models in Efficient and Authentic Global Gesture Generation from Audios
  8. 2023ArchiSound: Audio Generation with Diffusion