mlsanityData to 7 Oct 2026

Anticipatory Music Transformer

arXiv 2306.08620 ↗First posted 14 Jun 2023cs.SD cs.LG eess.AScited by 35

Not on the map. Why?

Abstract (arXiv)

We introduce anticipation: a method for constructing a controllable generative model of a temporal point process (the event process) conditioned asynchronously on realizations of a second, correlated process (the control process). We achieve this by interleaving sequences of events and controls, such that controls appear following stopping times in the event sequence. This work is motivated by problems arising in the control of symbolic music generation. We focus on infilling control tasks, whereby the controls are a subset of the events themselves, and conditional generation completes a sequence of events given the fixed control events. We train anticipatory infilling models using the large and diverse Lakh MIDI music dataset. These models match the performance of autoregressive models for prompted music generation, with the additional capability to perform infilling control tasks, including accompaniment. Human evaluators report that an anticipatory model produces accompaniments with similar musicality to even music composed by humans over a 20-second clip.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Builds on 75 of its 122 references that are in the map. First posted on arXiv 14 Jun 2023.

Builds on

75 of 122 references.

References · 75

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  7. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  8. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  9. 2020Longformer: The Long-Document Transformercited by 3,362
  10. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  11. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  12. 2019The Curious Case of Neural Text Degenerationcited by 2,881
38 more
  1. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  2. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  3. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  4. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  5. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  6. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  7. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  8. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  9. 2018Model Cards for Model Reportingcited by 1,068
  10. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  11. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  12. 2023MusicLM: Generating Music From Textcited by 442
  13. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 418
  14. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 415
  15. 2023Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Modelscited by 323
  16. 2022AudioGen: Textually Guided Audio Generationcited by 292
  17. 2022Efficient Training of Language Models to Fill in the Middlecited by 248
  18. 2022CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilitiescited by 245
  19. 2020Transformer Hawkes Processcited by 230
  20. 2018Music Transformercited by 207
  21. 2023Noise2Music: Text-conditioned Music Generation with Diffusion Modelscited by 173
  22. 2020A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarioscited by 170
  23. 2021Symbolic Music Generation with Diffusion Modelscited by 152
  24. 2017MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompanimentcited by 151
  25. 2022CM3: A Causal Masked Multimodal Model of the Internetcited by 142
  26. 2019Intensity-Free Learning of Temporal Point Processescited by 138
  27. 2020Enabling Language Models to Fill in the Blankscited by 121
  28. 2019Fully Neural Network based Model for General Temporal Point Processescited by 119
  29. 2023Foundation Models and Fair Usecited by 112
  30. 2021Compound Word Transformer: Learning to Compose Full-Song Music over Dynamic Directed Hypergraphscited by 109
  31. 2023On Provable Copyright Protection for Generative Modelscited by 97
  32. 2016DeepBach: a Steerable Model for Bach Chorales Generationcited by 90
  33. 2021MusicBERT: Symbolic Music Understanding with Large-Scale Pre-Trainingcited by 89
  34. 2020Shortformer: Better Language Modeling using Shorter Inputscited by 76
  35. 2021MT3: Multi-Task Multitrack Music Transcriptioncited by 70
  36. 2020PopMAG: Pop Music Accompaniment Generationcited by 68
  37. 2013A Deep and Tractable Density Estimatorcited by 67
  38. 2018This Time with Feeling: Learning Expressive Musical Performancecited by 67

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 8; 2025: 12; 2026: 12.

35 papers.

Sort:

By area:

  1. 2024Foundation Models for Music: A Surveycited by 34
  2. 2024Whole-Song Hierarchical Generation of Symbolic Music Using Cascaded Diffusion Modelscited by 24
  3. 2025Aligning Text-to-Music Evaluation with Human Preferencescited by 20
  4. 2024PDMX: A Large-Scale Public Domain MusicXML Dataset for Symbolic Music Processingcited by 15
  5. 2025Aria-MIDI: A Dataset of Piano MIDI Files for Symbolic Music Modelingcited by 10
  6. 2023Structured Multi-Track Accompaniment Arrangement via Style Prior Modellingcited by 9
  7. 2025Scaling Self-Supervised Representation Learning for Symbolic Piano Performancecited by 7
  8. 2024Composer's Assistant 2: Interactive Multi-Track MIDI Infilling with Fine-Grained User Controlcited by 5
  9. 2024PerTok: Expressive Encoding and Modeling of Symbolic Musical Ideas and Variationscited by 4
  10. 2025Moonbeam: A MIDI Foundation Model Using Both Absolute and Relative Music Attributescited by 4
  11. 2025ImprovNet -- Generating Controllable Musical Improvisations with Iterative Corruption Refinementcited by 3
  12. 2023Computational Copyright: Towards A Royalty Model for Music Generative AIcited by 2
23 more
  1. 2024Local deployment of large-scale music AI models on commodity hardwarecited by 2
  2. 2025PhraseVAE and PhraseLDM: Latent Diffusion for Full-Song Multitrack Symbolic Music Generationcited by 2
  3. 2026BEAT: Tokenizing and Generating Symbolic Music by Uniform Temporal Stepscited by 2
  4. 2026LiveBand: Live Accompaniment Generation in the Audio Domaincited by 2
  5. 2024Music Proofreading with RefinPaint: Where and How to Modify Compositions given Contextcited by 1
  6. 2024Generating High-quality Symbolic Music Using Fine-grained Discriminatorscited by 1
  7. 2025On the de-duplication of the Lakh MIDI datasetcited by 1
  8. 2025MuseTok: Symbolic Music Tokenization for Generation and Semantic Understandingcited by 1
  9. 2025MIDI-LLM: Improving Text-to-MIDI Music Generation via Adapting Large Language Modelscited by 1
  10. 2026ViTex: Visual Texture Control for Multi-Track Symbolic Music Generation via Discrete Diffusion Modelscited by 1
  11. 2026What's a Credit Worth? A Market Framework for Attribution-Aware Compensation in Generative Musiccited by 1
  12. 2023Unsupervised Lead Sheet Generation via Semantic Compressioncited by 0
  13. 2025TOMI: Transforming and Organizing Music Ideas for Multi-Track Compositions with Full-Song Structurecited by 0
  14. 2025TinyMusician: On-Device Music Generation with Knowledge Distillation and Mixed Precision Quantizationcited by 0
  15. 2025Supporting Creative Ownership through Deep Learning-Based Music Variationcited by 0
  16. 2026Structure-Aware Piano Accompaniment via Style Planning and Dataset-Aligned Pattern Retrievalcited by 0
  17. 2026Libretto: Giving LLM Agents a Sense of Musical Structurecited by 0
  18. 2026Verifier-Guided Twelve-Tone Composition: A Generate-Verify-Repair Harness for Symbolic Music Generationcited by 0
  19. 2026Learning Music Style for Piano Arrangement Through Cross-Modal Bootstrappingcited by 0
  20. 2026Equivariant Music Transformercited by 0
  21. 2026MI-MIDI: Mechanistic Interpretability of Text-to-MIDI Generation Models via Probing, Lenses and Steeringcited by 0
  22. 2026Which Constraints Are Missing? Ask the Verifier: Graded Rewards for Constraint-Following Music Generationcited by 0
  23. 2026JazzSAMBA: A Synchronous and Asynchronous Multi-take Band Audio Dataset of Jazz Standards for Live Music Modelscited by 0