mlsanityData to 7 Oct 2026

MusicMagus: Zero-Shot Text-to-Music Editing via Diffusion Models

arXiv 2402.06178 ↗First posted 9 Feb 2024cs.SD cs.AI cs.MMcited by 32

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Recent advances in text-to-music generation models have opened new avenues in musical creativity. However, music generation usually involves iterative refinements, and how to edit the generated music remains a significant challenge. This paper introduces a novel approach to the editing of music generated by such models, enabling the modification of specific attributes, such as genre, mood and instrument, while maintaining other aspects unchanged. Our method transforms text editing to textitlatent space manipulation while adding an extra constraint to enforce consistency. It seamlessly integrates with existing pretrained text-to-music diffusion models without requiring additional training. Experimental results demonstrate superior performance over both zero-shot and certain supervised baselines in style and timbre transfer evaluations. Additionally, we showcase the practical applicability of our approach in real-world music editing scenarios.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 40 of its 50 references that are in the map. First posted on arXiv 9 Feb 2024.

Builds on

40 of 50 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024DITTO: Diffusion Inference-Time T-Optimization for Music Generation22 Jan
  2. 2023Mustango: Toward Controllable Text-to-Music Generation14 Nov
  3. 2023InstructME: An Instruction Guided Music Edit And Remix Framework with Latent Diffusion Models28 Aug
  4. 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining10 Aug
  5. 2023JEN-1: Text-Guided Universal Music Generation with Omnidirectional Diffusion Models9 Aug
  6. 2023MusicLDM: Enhancing Novelty in Text-to-Music Generation Using Beat-Synchronous Mixup Strategies3 Aug
  7. 2023Polyffusion: A Diffusion Model for Polyphonic Score Generation with Internal and External Controls19 Jul
  8. 2023Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model24 Apr
  9. 2023AUDIT: Audio Editing by Following Instructions with Latent Diffusion Models3 Apr

Other references · 31

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  7. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  8. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  9. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 707
  10. 2023Zero-shot Image-to-Image Translationcited by 503
  11. 2023Simple and Controllable Music Generationcited by 480
  12. 2023MusicLM: Generating Music From Textcited by 442
19 more
  1. 2022AudioGen: Textually Guided Audio Generationcited by 292
  2. 2022Masked Autoencoders that Listencited by 271
  3. 2018Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Datasetcited by 215
  4. 2020POP909: A Pop-song Dataset for Music Arrangement Generationcited by 83
  5. 2023Music ControlNet: Multiple Time-varying Controls for Music Generationcited by 82
  6. 2023Music Understanding LLaMA: Advancing Text-to-Music Generation with Question Answering and Captioningcited by 77
  7. 2023The Song Describer Dataset: a Corpus of Audio Captions for Music-and-Language Evaluationcited by 68
  8. 2024Instruct-Imagen: Image Generation with Multi-modal Instructioncited by 65
  9. 2023Efficient Neural Music Generationcited by 50
  10. 2023MuseCoco: Generating Symbolic Music from Textcited by 50
  11. 2020HiFi-GAN: High-Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networkscited by 40
  12. 2023StemGen: A music generation model that listenscited by 32
  13. 2023M2UGen: Multi-modal Music Understanding and Generation with the Power of Large Language Modelscited by 24
  14. 2021AccoMontage: Accompaniment Arrangement via Phrase Selection and Style Transfercited by 22
  15. 2023Content-based Controls For Music Large Language Modelingcited by 19
  16. 2023JEN-1 Composer: A Unified Framework for High-Fidelity Multi-Track Music Generationcited by 17
  17. 2018Music Style Transfer: A Position Papercited by 16
  18. 2023Loop Copilot: Conducting AI Ensembles for Music Generation and Iterative Editingcited by 15
  19. 2024Efficient Parallel Audio Generation using Group Masked Language Modelingcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 15; 2026: 2.

32 papers.

Sort:

By area:

  1. 2024Zero-Shot Unsupervised and Text-Based Audio Editing Using DDPM Inversioncited by 58
  2. 2024Diff-A-Riff: Musical Accompaniment Co-creation via Latent Diffusion Modelscited by 33
  3. 2024Instruct-MusicGen: Unlocking Text-to-Music Editing for Music Language Models via Instruction Tuningcited by 15
  4. 2024High Fidelity Text-Guided Music Editing via Single-Stage Flow Matchingcited by 13
  5. 2024Editing Music with Melody and Text: Using ControlNet for Diffusion Transformercited by 13
  6. 2024Audio Prompt Adapter: Unleashing Music Editing Abilities for Text-to-Music with Lightweight Finetuningcited by 12
  7. 2024MusiConGen: Rhythm and Chord Control for Transformer-Based Text-to-Music Generationcited by 11
  8. 2024Improving Musical Accompaniment Co-creation via Diffusion Transformerscited by 11
  9. 2024Latent Diffusion Bridges for Unsupervised Musical Audio Timbre Transfercited by 9
  10. 2025AudioMorphix: Training-free audio editing with diffusion probabilistic modelscited by 6
  11. 2025SonicMaster: Towards Controllable All-in-One Music Restoration and Masteringcited by 6
  12. 2024Futga: Towards Fine-grained Music Understanding through Temporally-enhanced Generative Augmentationcited by 5
20 more
  1. 2024SoundMorpher: Perceptually-Uniform Sound Morphing with Diffusion Modelcited by 3
  2. 2025LiLAC: A Lightweight Latent ControlNet for Musical Audio Generationcited by 3
  3. 2025GD-Retriever: Controllable Generative Text-Music Retrieval with Diffusion Modelscited by 3
  4. 2024Unsupervised Disentanglement of Content and Style via Variance-Invariance Constraintscited by 2
  5. 2025WhisQ: Cross-Modal Representation Learning for Text-to-Music MOS Predictioncited by 2
  6. 2025Via Score to Performance: Efficient Human-Controllable Long Song Generation with Bar-Level Symbolic Notationcited by 2
  7. 2025Neural Speech Extraction with Human Feedbackcited by 2
  8. 2026TADA! Tuning Audio Diffusion Models through Activation Steeringcited by 2
  9. 2024The Interpretation Gap in Text-to-Music Generation Modelscited by 1
  10. 2024Prevailing Research Areas for Music AI in the Era of Foundation Modelscited by 1
  11. 2025RASA: Replace Anyone, Say Anything -- A Training-Free Framework for Audio-Driven and Universal Portrait Video Editingcited by 1
  12. 2025Not that Groove: Zero-Shot Symbolic Music Editingcited by 1
  13. 2025Noise-to-Notes: Diffusion-based Generation and Refinement for Automatic Drum Transcriptioncited by 1
  14. 2025AI-Assisted Music Production: A User Study on Text-to-Music Modelscited by 1
  15. 2025MIDI-LLM: Improving Text-to-MIDI Music Generation via Adapting Large Language Modelscited by 1
  16. 2024Annotation-Free MIDI-to-Audio Synthesis via Concatenative Synthesis and Generative Refinementcited by 0
  17. 2025Mozualization: Crafting Music and Visual Representation with Multimodal AIcited by 0
  18. 2025BNMusic: Blending Environmental Noises into Personalized Musiccited by 0
  19. 2025MusRec: Zero-Shot Text-to-Music Editing via Rectified Flow and Diffusion Transformerscited by 0
  20. 2026FlowSonic: Stable Zero-Shot Music Editing via High-Order Trajectory Integrationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Diff-TONE: Timestep Optimization for iNstrument Editing in Text-to-Music Diffusion Models
  2. 2025SteerMusic: Enhanced Musical Consistency for Zero-shot Text-guided and Personalized Music Editing
  3. 2025Melodia: Training-Free Music Editing Guided by Attention Probing in Diffusion Models
  4. 2025Generation of Musical Timbres using a Text-Guided Diffusion Model
  5. 2026Diffusion Timbre Transfer Via Mutual Information Guided Inpainting
  6. 2024Music Style Transfer with Time-Varying Inversion of Diffusion Models
  7. 2024Multi-Track MusicLDM: Towards Versatile Music Generation with Latent Diffusion Model
  8. 2024Combining audio control and style transfer using latent diffusion