mlsanityData to 7 Oct 2026

Loop Copilot: Conducting AI Ensembles for Music Generation and Iterative Editing

arXiv 2310.12404 ↗First posted 19 Oct 2023cs.SD cs.CL cs.HCcited by 15

Not on the map. Why?

Abstract (arXiv)

Creating music is iterative, requiring varied methods at each stage. However, existing AI music systems fall short in orchestrating multiple subsystems for diverse needs. To address this gap, we introduce Loop Copilot, a novel system that enables users to generate and iteratively refine music through an interactive, multi-round dialogue interface. The system uses a large language model to interpret user intentions and select appropriate AI models for task execution. Each backend model is specialized for a specific task, and their outputs are aggregated to meet the user's requirements. To ensure musical coherence, essential attributes are maintained in a centralized table. We evaluate the effectiveness of the proposed system through semi-structured interviews and questionnaires, highlighting its utility not only in facilitating music creation but also its potential for broader applications.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 30 of its 61 references that are in the map. First posted on arXiv 19 Oct 2023.

Builds on

30 of 61 references.

References · 30

  1. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
  2. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 659
  3. 2020Jukebox: A Generative Model for Musiccited by 535
  4. 2023Simple and Controllable Music Generationcited by 480
  5. 2023MusicLM: Generating Music From Textcited by 442
  6. 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretrainingcited by 343
  7. 2023AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Headcited by 264
  8. 2021Symbolic Music Generation with Diffusion Modelscited by 152
  9. 2022Hybrid Transformers for Music Source Separationcited by 134
  10. 2023MusicLDM: Enhancing Novelty in Text-to-Music Generation Using Beat-Synchronous Mixup Strategiescited by 94
  11. 2021TorchAudio: Building Blocks for Audio and Speech Processingcited by 83
  12. 2023LP-MusicCaps: LLM-Based Pseudo Music Captioningcited by 83
18 more
  1. 2023AUDIT: Audio Editing by Following Instructions with Latent Diffusion Modelscited by 81
  2. 2023VampNet: Music Generation via Masked Acoustic Token Modelingcited by 57
  3. 2020A Comprehensive Survey on Deep Music Generation: Multi-level Representations, Algorithms, Evaluations, and Future Directionscited by 56
  4. 2020CoCon: A Self-Supervised Approach for Controlled Text Generationcited by 47
  5. 2020Music FaderNets: Controllable Music Generation Based On High-Level Features via Low-Level Feature Modellingcited by 45
  6. 2019Counterpoint by Convolutioncited by 42
  7. 2020SongMASS: Automatic Song Writing with Pre-training and Alignment Constraintcited by 41
  8. 2020Learning Interpretable Representation for Controllable Polyphonic Music Generationcited by 40
  9. 2023Polyffusion: A Diffusion Model for Polyphonic Score Generation with Internal and External Controlscited by 35
  10. 2019Deep Music Analogy Via Latent Representation Disentanglementcited by 31
  11. 2021MIDI-DDSP: Detailed Control of Musical Performance via Hierarchical Modelingcited by 31
  12. 2021Controllable deep melody generation via hierarchical music structure representationcited by 30
  13. 2020AI Song Contest: Human-AI Co-Creation in Songwritingcited by 25
  14. 2023InstructME: An Instruction Guided Music Edit And Remix Framework with Latent Diffusion Modelscited by 23
  15. 2021AccoMontage: Accompaniment Arrangement via Phrase Selection and Style Transfercited by 22
  16. 2023MusicAgent: An AI Agent for Music Understanding and Generation with Large Language Modelscited by 20
  17. 2019Musical Composition Style Transfer via Disentangled Timbre Representationscited by 11
  18. 2022AccoMontage2: A Complete Harmonization and Accompaniment Arrangement Systemcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 12; 2025: 1; 2026: 2.

15 papers.

Sort:

By area:

  1. 2024MMedAgent: Learning to Use Medical Tools with Multi-modal Agentcited by 108
  2. 2024Large Multimodal Agents: A Surveycited by 99
  3. 2024ComposerX: Multi-Agent Symbolic Music Composition with LLMscited by 40
  4. 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
  5. 2024Foundation Models for Music: A Surveycited by 34
  6. 2024MusicMagus: Zero-Shot Text-to-Music Editing via Diffusion Modelscited by 32
  7. 2024Taming Data and Transformers for Audio Generationcited by 28
  8. 2024Prompt-guided Precise Audio Editing with Diffusion Modelscited by 18
  9. 2024Instruct-MusicGen: Unlocking Text-to-Music Editing for Music Language Models via Instruction Tuningcited by 15
  10. 2024Human-Centered LLM-Agent User Interface: A Position Papercited by 3
  11. 2025Agentic Design Review Systemcited by 2
  12. 2024R4: Reinforced Retriever-Reorder-Responder for Retrieval-Augmented Large Language Modelscited by 1
3 more
  1. 2024The Interpretation Gap in Text-to-Music Generation Modelscited by 1
  2. 2026A Comprehensive Study of Implementation Bugs in Multi-modal Agentscited by 0
  3. 2026A Production-Oriented Framework for Evaluation of SFX Generationcited by 0