# Learning to Groove with Inverse Sequence Transformations

arXiv: https://arxiv.org/abs/1905.06118 · HTML: https://mlsanity.com/paper/1905.06118/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 May 2019.

## Built on by (most cited first, 30 of 34)

- [Beat this! Accurate beat tracking without DBN postprocessing](https://mlsanity.com/paper/2407.21658/) (2024)
- [V2Meow: Meowing to the Visual Beat via Video-to-Music Generation](https://mlsanity.com/paper/2305.06594/) (2023)
- [The Ethical Implications of Generative Audio Models: A Systematic Literature Review](https://mlsanity.com/paper/2307.05527/) (2023)
- [JukeDrummer: Conditional Beat-aware Audio-domain Drum Accompaniment Generation via Transformer VQ-VAE](https://mlsanity.com/paper/2210.06007/) (2022)
- [A Review of Intelligent Music Generation Systems](https://mlsanity.com/paper/2211.09124/) (2022)
- [A Survey of AI Music Generation Tools and Models](https://mlsanity.com/paper/2308.12982/) (2023)
- [Conditional Drums Generation using Compound Word Representations](https://mlsanity.com/paper/2202.04464/) (2022)
- [Modeling Recommender Ecosystems: Research Challenges at the Intersection of Mechanism Design, Reinforcement Learning and Generative Models](https://mlsanity.com/paper/2309.06375/) (2023)
- [The GigaMIDI Dataset with Features for Expressive Music Performance Detection](https://mlsanity.com/paper/2502.17726/) (2025)
- [Downbeat Tracking with Tempo-Invariant Convolutional Neural Networks](https://mlsanity.com/paper/2102.02282/) (2021)
- [Toward Deep Drum Source Separation](https://mlsanity.com/paper/2312.09663/) (2023)
- [A Comprehensive Survey on Generative AI for Video-to-Music Generation](https://mlsanity.com/paper/2502.12489/) (2025)
- [Moonbeam: A MIDI Foundation Model Using Both Absolute and Relative Music Attributes](https://mlsanity.com/paper/2505.15559/) (2025)
- [Language Models are Drummers: Drum Composition with Natural Language Pre-Training](https://mlsanity.com/paper/2301.01162/) (2023)
- [The Inverse Drum Machine: Source Separation Through Joint Transcription and Analysis-by-Synthesis](https://mlsanity.com/paper/2505.03337/) (2025)
- [Generating Coherent Drum Accompaniment With Fills And Improvisations](https://mlsanity.com/paper/2209.00291/) (2022)
- [A Survey of Music Generation in the Context of Interaction](https://mlsanity.com/paper/2402.15294/) (2024)
- [Applications and Advances of Artificial Intelligence in Music Generation:A Review](https://mlsanity.com/paper/2409.03715/) (2024)
- [DARC: Drum accompaniment generation with fine-grained rhythm control](https://mlsanity.com/paper/2601.02357/) (2026)
- [SonicBench: Dissecting the Physical Perception Bottleneck in Large Audio Language Models](https://mlsanity.com/paper/2601.11039/) (2026)
- [Flat Latent Manifolds for Human-machine Co-creation of Music](https://mlsanity.com/paper/2202.12243/) (2022)
- [Analyzing and reducing the synthetic-to-real transfer gap in Music Information Retrieval: the task of automatic drum transcription](https://mlsanity.com/paper/2407.19823/) (2024)
- [Exploring Situated Stabilities of a Rhythm Generation System through Variational Cross-Examination](https://mlsanity.com/paper/2509.05145/) (2025)
- [Drum Synthesis from Expressive Drum Grids via Neural Audio Codecs](https://mlsanity.com/paper/2605.10281/) (2026)
- [Proceedings of the 4th International Workshop on Reading Music Systems](https://mlsanity.com/paper/2211.13285/) (2022)
- [Notochord: a Flexible Probabilistic Model for Real-Time MIDI Performance](https://mlsanity.com/paper/2403.12000/) (2024)
- [AlignBeat: A Latent Variable Model for Multi-Class Beat Tracking from Partially Labeled Data](https://mlsanity.com/paper/2510.14391/) (2025)
- [On the Joint Minimization of Regularization Loss Functions in Deep Variational Bayesian Methods for Attribute-Controlled Symbolic Music Generation](https://mlsanity.com/paper/2511.07118/) (2025)
- [Can pre-trained Deep Learning models predict groove ratings?](https://mlsanity.com/paper/2603.27237/) (2026)
- [A Cold Diffusion Approach for Percussive Dereverberation](https://mlsanity.com/paper/2605.10256/) (2026)

## Abstract (arXiv)

We explore models for translating abstract musical ideas (scores, rhythms) into expressive performances using Seq2Seq and recurrent Variational Information Bottleneck (VIB) models. Though Seq2Seq models usually require painstakingly aligned corpora, we show that it is possible to adapt an approach from the Generative Adversarial Network (GAN) literature (e.g. Pix2Pix (Isola et al., 2017) and Vid2Vid (Wang et al. 2018a)) to sequences, creating large volumes of paired data by performing simple transformations and training generative models to plausibly invert these transformations. Music, and drumming in particular, provides a strong test case for this approach because many common transformations (quantization, removing voices) have clear semantics, and models for learning to invert them have real-world applications. Focusing on the case of drum set players, we create and release a new dataset for this purpose, containing over 13 hours of recordings by professional drummers aligned with fine-grained timing and dynamics information. We also explore some of the creative potential of these models, including demonstrating improvements on state-of-the-art methods for Humanization (instantiating a performance from a musical score).