# The Inverse Drum Machine: Source Separation Through Joint Transcription and Analysis-by-Synthesis

arXiv: https://arxiv.org/abs/2505.03337 · HTML: https://mlsanity.com/paper/2505.03337/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 62 references that are in the map.
- First posted on arXiv 6 May 2025.

## Builds on (most cited first, 30 of 31)

- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [DDSP: Differentiable Digital Signal Processing](https://mlsanity.com/paper/2001.04643/) (2020)
- [Onsets and Frames: Dual-Objective Piano Transcription](https://mlsanity.com/paper/1710.11153/) (2017)
- [Hybrid Spectrogram and Waveform Source Separation](https://mlsanity.com/paper/2111.03600/) (2021)
- [Neural source-filter waveform models for statistical parametric speech synthesis](https://mlsanity.com/paper/1904.12088/) (2019)
- [Music Demixing Challenge 2021](https://mlsanity.com/paper/2108.13559/) (2021)
- [CRASH: Raw Audio Score-based Generative Modeling for Controllable High-resolution Drum Sound Synthesis](https://mlsanity.com/paper/2106.07431/) (2021)
- [DrumGAN: Synthesis of Drum Sounds With Timbral Feature Conditioning Using Generative Adversarial Networks](https://mlsanity.com/paper/2008.12073/) (2020)
- [Learning to Groove with Inverse Sequence Transformations](https://mlsanity.com/paper/1905.06118/) (2019)
- [A Review of Differentiable Digital Signal Processing for Music & Speech Synthesis](https://mlsanity.com/paper/2308.15422/) (2023)
- [Simultaneous Separation and Transcription of Mixtures with Multiple Polyphonic and Percussive Instruments](https://mlsanity.com/paper/1910.12621/) (2019)
- [Improving Perceptual Quality of Drum Transcription with the Expanded Groove MIDI Dataset](https://mlsanity.com/paper/2004.00188/) (2020)
- [Sinusoidal Frequency Estimation by Gradient Descent](https://mlsanity.com/paper/2210.14476/) (2022)
- [Objective Measures of Perceptual Audio Quality Reviewed: An Evaluation of Their Application Domain Dependence](https://mlsanity.com/paper/2110.11438/) (2021)
- [The Effect of Spectrogram Reconstruction on Automatic Music Transcription: An Alternative Approach to Improve Transcription Accuracy](https://mlsanity.com/paper/2010.09969/) (2020)
- [Neural Drum Machine : An Interactive System for Real-time Synthesis of Drum Sounds](https://mlsanity.com/paper/1907.02637/) (2019)
- [Neural Percussive Synthesis Parameterised by High-Level Timbral Features](https://mlsanity.com/paper/1911.11853/) (2019)
- [Unsupervised Harmonic Parameter Estimation Using Differentiable DSP and Spectral Optimal Transport](https://mlsanity.com/paper/2312.14507/) (2023)
- [Deep Unsupervised Drum Transcription](https://mlsanity.com/paper/1906.03697/) (2019)
- [Towards multi-instrument drum transcription](https://mlsanity.com/paper/1806.06676/) (2018)
- [Blind estimation of audio effects using an auto-encoder approach and differentiable digital signal processing](https://mlsanity.com/paper/2310.11781/) (2023)
- [wav2shape: Hearing the Shape of a Drum Machine](https://mlsanity.com/paper/2007.10299/) (2020)
- [Unsupervised Music Source Separation Using Differentiable Parametric Source Models](https://mlsanity.com/paper/2201.09592/) (2022)
- [Differentiable Digital Signal Processing Mixture Model for Synthesis Parameter Extraction from Mixture of Harmonic Sounds](https://mlsanity.com/paper/2202.00200/) (2022)
- [Toward Deep Drum Source Separation](https://mlsanity.com/paper/2312.09663/) (2023)
- [Differentiable Modelling of Percussive Audio with Transient and Spectral Synthesis](https://mlsanity.com/paper/2309.06649/) (2023)
- [A fully differentiable model for unsupervised singing voice separation](https://mlsanity.com/paper/2401.16837/) (2024)

## Built on by (most cited first, 3 of 3)

- [Break-the-Beat! Controllable MIDI-to-Drum Audio Synthesis](https://mlsanity.com/paper/2605.14555/) (2026)
- [Separate-and-Detect: Unified Drum Transcription and Stem Generation via Latent Diffusion](https://mlsanity.com/paper/2608.01093/) (2026)
- [TCPα: Margin-Controlled Confidence estimation for reliable Music Information Retrieval](https://mlsanity.com/paper/2608.20326/) (2026)

## Abstract (arXiv)

We present the Inverse Drum Machine, a novel approach to Drum Source Separation that leverages an analysis-by-synthesis framework combined with deep learning. Unlike recent supervised methods that require isolated stem recordings for training, our approach is trained on drum mixtures with only transcription annotations. IDM integrates Automatic Drum Transcription and One-shot Drum Sample Synthesis, jointly optimizing these tasks in an end-to-end manner. By convolving synthesized one-shot samples with estimated onsets, akin to a drum machine, we reconstruct the individual drum stems and train a Deep Neural Network on the reconstruction of the mixture. Experiments on the StemGMD dataset demonstrate that IDM achieves separation quality comparable to state-of-the-art supervised methods that require isolated stems data.