mlsanityData to 7 Oct 2026

Live Music Models

arXiv 2508.04651 ↗First posted 6 Aug 2025cs.SD cs.HC cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

We introduce a new class of generative models for music called live music models that produce a continuous stream of music in real-time with synchronized user control. We release Magenta RealTime, an open-weights live music model that can be steered using text or audio prompts to control acoustic style. On automatic metrics of music quality, Magenta RealTime outperforms other open-weights music generation models, despite using fewer parameters and offering first-of-its-kind live generation capabilities. We also release Lyria RealTime, an API-based model with extended controls, offering access to our most powerful model with wide prompt coverage. These models demonstrate a new paradigm for AI-assisted music creation that emphasizes human-in-the-loop interaction for live music performance.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 43 of its 63 references that are in the map. First posted on arXiv 6 Aug 2025.

Builds on

43 of 63 references.

References · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2022Classifier-Free Diffusion Guidancecited by 5,717
  5. 2017Neural Discrete Representation Learningcited by 4,650
  6. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  7. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  8. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  9. 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
  10. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
  11. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 536
  12. 2020Jukebox: A Generative Model for Musiccited by 535
31 more
  1. 2023MusicLM: Generating Music From Textcited by 442
  2. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 423
  3. 2024Stable Audio Opencited by 192
  4. 2022Scaling Up Models and Data with textttt5x and textttseqiocited by 187
  5. 2024Fast Timing-Conditioned Latent Audio Diffusioncited by 178
  6. 2020DDSP: Differentiable Digital Signal Processingcited by 177
  7. 2023UniAudio: An Audio Foundation Model Toward Universal Audio Generationcited by 127
  8. 2023MusicLDM: Enhancing Novelty in Text-to-Music Generation Using Beat-Synchronous Mixup Strategiescited by 94
  9. 2023Stay on topic with Classifier-Free Guidancecited by 90
  10. 2024Long-form music generation with latent diffusioncited by 88
  11. 2022MuLan: A Joint Embedding of Music Audio and Natural Languagecited by 85
  12. 2024DITTO: Diffusion Inference-Time T-Optimization for Music Generationcited by 73
  13. 2021RAVE: A variational autoencoder for fast and high-quality neural audio synthesiscited by 72
  14. 2023The Song Describer Dataset: a Corpus of Audio Captions for Music-and-Language Evaluationcited by 68
  15. 2025YuE: Scaling Open Foundation Models for Long-Form Music Generationcited by 68
  16. 2018The challenge of realistic music generation: modelling raw audio at scalecited by 63
  17. 2021Codified audio language modeling learns useful representations for music information retrievalcited by 61
  18. 2023SingSong: Generating musical accompaniments from singingcited by 54
  19. 2025ACE-Step: A Step Towards Music Generation Foundation Modelcited by 54
  20. 2024Towards audio language modeling -- an overviewcited by 40
  21. 2017Latent Constraints: Learning to Generate Conditionally from Unconditional Generative Modelscited by 39
  22. 2023Controllable Music Production with Diffusion Models and Guidance Gradientscited by 29
  23. 2025SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generationcited by 29
  24. 2021KUIELab-MDX-Net: A Two-Stream Neural Network for Music Demixingcited by 24
  25. 2020RL-Duet: Online Music Accompaniment Generation Using Deep Reinforcement Learningcited by 19
  26. 2023Content-based Controls For Music Large Language Modelingcited by 19
  27. 2025Fast Text-to-Audio Generation with Adversarial Post-Trainingcited by 14
  28. 2018Piano Geniecited by 11
  29. 2022Beat Transformer: Demixed Beat and Downbeat Tracking with Dilated Self-Attentioncited by 9
  30. 2025Adaptive Accompaniment with ReaLchordscited by 4
  31. 2024Local deployment of large-scale music AI models on commodity hardwarecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 8.

9 papers.

  1. 2026CMI-RewardBench: Evaluating Music Reward Models with Compositional Multimodal Instructioncited by 5
  2. 2026Towards Real-Time Human-AI Musical Co-Performance: Accompaniment Generation with Latent Diffusion Models and MAX/MSPcited by 2
  3. 2026LiveBand: Live Accompaniment Generation in the Audio Domaincited by 2
  4. 2026Live Music Diffusion Models: Efficient Fine-Tuning and Post-Training of Interactive Diffusion Music Generatorscited by 1
  5. 2026What's a Credit Worth? A Market Framework for Attribution-Aware Compensation in Generative Musiccited by 1
  6. 2025The Ghost in the Keys: A Disklavier Demo for Human-AI Musical Co-Creativitycited by 0
  7. 2026Auditing Training Data in Generative Music Models via Black-Box Membership Inferencecited by 0
  8. 2026Noisy-Channel Minimum Bayes Risk Decodingcited by 0
  9. 2026TrueMuse: A Benchmark for Data Attribution in Text-to-Music Modelscited by 0