# CRASH: Raw Audio Score-based Generative Modeling for Controllable High-resolution Drum Sound Synthesis

arXiv: https://arxiv.org/abs/2106.07431 · HTML: https://mlsanity.com/paper/2106.07431/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 28 of its 33 references that are in the map.
- First posted on arXiv 14 Jun 2021.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 28 of 28)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [WaveGrad: Estimating Gradients for Waveform Generation](https://mlsanity.com/paper/2009.00713/) (2020)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Designing an Encoder for StyleGAN Image Manipulation](https://mlsanity.com/paper/2102.02766/) (2021)
- [WaveGlow: A Flow-based Generative Network for Speech Synthesis](https://mlsanity.com/paper/1811.00002/) (2018)
- [Adversarial Audio Synthesis](https://mlsanity.com/paper/1802.04208/) (2018)
- [Fr'echet Audio Distance: A Metric for Evaluating Music Enhancement Algorithms](https://mlsanity.com/paper/1812.08466/) (2018)
- [GANSynth: Adversarial Neural Audio Synthesis](https://mlsanity.com/paper/1902.08710/) (2019)
- [SampleRNN: An Unconditional End-to-End Neural Audio Generation Model](https://mlsanity.com/paper/1612.07837/) (2016)
- [MelNet: A Generative Model for Audio in the Frequency Domain](https://mlsanity.com/paper/1906.01083/) (2019)
- [A Spectral Energy Distance for Parallel Speech Synthesis](https://mlsanity.com/paper/2008.01160/) (2020)
- [DrumGAN: Synthesis of Drum Sounds With Timbral Feature Conditioning Using Generative Adversarial Networks](https://mlsanity.com/paper/2008.12073/) (2020)
- [Conditioned-U-Net: Introducing a Control Mechanism in the U-Net for Multiple Source Separations](https://mlsanity.com/paper/1907.01277/) (2019)
- [Neural Drum Machine : An Interactive System for Real-time Synthesis of Drum Sounds](https://mlsanity.com/paper/1907.02637/) (2019)

## Built on by (most cited first, 30 of 37)

- [Fast Timing-Conditioned Latent Audio Diffusion](https://mlsanity.com/paper/2402.04825/) (2024)
- [Protein structure generation via folding diffusion](https://mlsanity.com/paper/2209.15611/) (2022)
- [Long-form music generation with latent diffusion](https://mlsanity.com/paper/2404.10301/) (2024)
- [A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?](https://mlsanity.com/paper/2303.11717/) (2023)
- [Universal Speech Enhancement with Score-based Diffusion](https://mlsanity.com/paper/2206.03065/) (2022)
- [A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AI](https://mlsanity.com/paper/2303.13336/) (2023)
- [Multi-Source Diffusion Models for Simultaneous Music Generation and Separation](https://mlsanity.com/paper/2302.02257/) (2023)
- [Solving Audio Inverse Problems with a Diffusion Model](https://mlsanity.com/paper/2210.15228/) (2022)
- [Enhancing Deep Reinforcement Learning: A Tutorial on Generative Diffusion Models in Network Optimization](https://mlsanity.com/paper/2308.05384/) (2023)
- [Scaling Transformers for Low-Bitrate High-Quality Speech Coding](https://mlsanity.com/paper/2411.19842/) (2024)
- [Multi-instrument Music Synthesis with Spectrogram Diffusion](https://mlsanity.com/paper/2206.05408/) (2022)
- [Foundation Models for Music: A Survey](https://mlsanity.com/paper/2408.14340/) (2024)
- [Diff-A-Riff: Musical Accompaniment Co-creation via Latent Diffusion Models](https://mlsanity.com/paper/2406.08384/) (2024)
- [Full-band General Audio Synthesis with Score-based Diffusion](https://mlsanity.com/paper/2210.14661/) (2022)
- [Controllable Music Production with Diffusion Models and Guidance Gradients](https://mlsanity.com/paper/2311.00613/) (2023)
- [T-FOLEY: A Controllable Waveform-Domain Diffusion Model for Temporal-Event-Guided Foley Sound Synthesis](https://mlsanity.com/paper/2401.09294/) (2024)
- [The Ethical Implications of Generative Audio Models: A Systematic Literature Review](https://mlsanity.com/paper/2307.05527/) (2023)
- [Protein Language Models and Structure Prediction: Connection and Progression](https://mlsanity.com/paper/2211.16742/) (2022)
- [Evaluating generative audio systems and their metrics](https://mlsanity.com/paper/2209.00130/) (2022)
- [Detecting Out-Of-Distribution Earth Observation Images with Diffusion Models](https://mlsanity.com/paper/2404.12667/) (2024)
- [Improving Musical Accompaniment Co-creation via Diffusion Transformers](https://mlsanity.com/paper/2410.23005/) (2024)
- [UnDiff: Unsupervised Voice Restoration with Unconditional Diffusion Model](https://mlsanity.com/paper/2306.00721/) (2023)
- [Timbre transfer using image-to-image denoising diffusion implicit models](https://mlsanity.com/paper/2307.04586/) (2023)
- [Msanii: High Fidelity Music Synthesis on a Shoestring Budget](https://mlsanity.com/paper/2301.06468/) (2023)
- [Zero-Shot Duet Singing Voices Separation with Diffusion Models](https://mlsanity.com/paper/2311.07345/) (2023)
- [Differentiable Digital Signal Processing Mixture Model for Synthesis Parameter Extraction from Mixture of Harmonic Sounds](https://mlsanity.com/paper/2202.00200/) (2022)
- [DrumGAN VST: A Plugin for Drum Sound Analysis/Synthesis With Autoencoding Generative Adversarial Networks](https://mlsanity.com/paper/2206.14723/) (2022)
- [Realistic Gramophone Noise Synthesis using a Diffusion Model](https://mlsanity.com/paper/2206.06259/) (2022)
- [MambaFoley: Foley Sound Generation using Selective State-Space Models](https://mlsanity.com/paper/2409.09162/) (2024)
- [The Inverse Drum Machine: Source Separation Through Joint Transcription and Analysis-by-Synthesis](https://mlsanity.com/paper/2505.03337/) (2025)

## Abstract (arXiv)

In this paper, we propose a novel score-base generative model for unconditional raw audio synthesis. Our proposal builds upon the latest developments on diffusion process modeling with stochastic differential equations, which already demonstrated promising results on image generation. We motivate novel heuristics for the choice of the diffusion processes better suited for audio generation, and consider the use of a conditional U-Net to approximate the score function. While previous approaches on diffusion models on audio were mainly designed as speech vocoders in medium resolution, our method termed CRASH (Controllable Raw Audio Synthesis with High-resolution) allows us to generate short percussive sounds in 44.1kHz in a controllable way. Through extensive experiments, we showcase on a drum sound generation task the numerous sampling schemes offered by our method (unconditional generation, deterministic generation, inpainting, interpolation, variations, class-conditional sampling) and propose the class-mixing sampling, a novel way to generate "hybrid" sounds. Our proposed method closes the gap with GAN-based methods on raw audio, while offering more flexible generation capabilities with lighter and easier-to-train models.