mlsanityData to 7 Oct 2026

High Fidelity Text-Guided Music Editing via Single-Stage Flow Matching

arXiv 2407.03648 ↗First posted 4 Jul 2024eess.AS cs.SDcited by 13

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

We introduce MelodyFlow, an efficient text-controllable high-fidelity music generation and editing model. It operates on continuous latent representations from a low frame rate 48 kHz stereo variational auto encoder codec. Based on a diffusion transformer architecture trained on a flow-matching objective the model can edit diverse high quality stereo samples of variable duration, with simple text descriptions. We adapt the ReNoise latent inversion method to flow matching and compare it with the original implementation and naive denoising diffusion implicit model (DDIM) inversion on a variety of music editing prompts. Our results indicate that our latent inversion outperforms both ReNoise and DDIM for zero-shot test-time text-guided editing on several objective metrics. Subjective evaluations exhibit a substantial improvement over previous state of the art for music editing. Code and model weights will be publicly made available. Samples are available at https://melodyflow.github.io.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 46 of its 52 references that are in the map. First posted on arXiv 4 Jul 2024.

Builds on

46 of 52 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024Zero-Shot Unsupervised and Text-Based Audio Editing Using DDPM Inversion15 Feb
  2. 2024MusicMagus: Zero-Shot Text-to-Music Editing via Diffusion Models9 Feb
  3. 2024DITTO: Diffusion Inference-Time T-Optimization for Music Generation22 Jan
  4. 2023Audiobox: Unified Audio Generation with Natural Language Prompts25 Dec
  5. 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining10 Aug
  6. 2023JEN-1: Text-Guided Universal Music Generation with Omnidirectional Diffusion Models9 Aug
  7. 2023Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model24 Apr
  8. 2023AUDIT: Audio Editing by Following Instructions with Latent Diffusion Models3 Apr
  9. 2023Noise2Music: Text-conditioned Music Generation with Diffusion Models8 Feb
  10. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Models29 Jan
  11. 2023Mousai: Text-to-Music Generation with Long-Context Latent Diffusion27 Jan

Other references · 35

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2022Flow Matching for Generative Modelingcited by 5,426
  5. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  6. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  7. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  8. 2023Improving and generalizing flow-based generative models with minibatch optimal transportcited by 835
  9. 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
  10. 2016CNN Architectures for Large-Scale Audio Classificationcited by 701
  11. 2022High Fidelity Neural Audio Compressioncited by 673
  12. 2023PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallelcited by 641
23 more
  1. 2023Zero-shot Image-to-Image Translationcited by 503
  2. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 498
  3. 2023Simple and Controllable Music Generationcited by 480
  4. 2023MusicLM: Generating Music From Textcited by 442
  5. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 423
  6. 2018SDR - half-baked or well done?cited by 420
  7. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
  8. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentationcited by 293
  9. 2022AudioGen: Textually Guided Audio Generationcited by 292
  10. 2022Masked Autoencoders that Listencited by 271
  11. 2023Multisample Flow Matching: Straightening Flows with Minibatch Couplingscited by 267
  12. 2023An Edit Friendly DDPM Noise Space: Inversion and Manipulationscited by 243
  13. 2021Efficient Training of Audio Transformers with Patchoutcited by 222
  14. 2024Fast Timing-Conditioned Latent Audio Diffusioncited by 178
  15. 2021Taming Visually Guided Sound Generationcited by 136
  16. 2024ReNoise: Real Image Inversion Through Iterative Noisingcited by 127
  17. 2023DiffiT: Diffusion Vision Transformers for Image Generationcited by 104
  18. 2022MuLan: A Joint Embedding of Music Audio and Natural Languagecited by 85
  19. 2023Music ControlNet: Multiple Time-varying Controls for Music Generationcited by 82
  20. 2023VampNet: Music Generation via Masked Acoustic Token Modelingcited by 57
  21. 2020Automatic multitrack mixing with a differentiable mixing console of neural audio effectscited by 27
  22. 2024Arrange, Inpaint, and Refine: Steerable Long-term Music Audio Generation and Editing via Content-based Controlscited by 17
  23. 2019Perceptual Loss Function for Neural Modelling of Audio Systemscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 5.

13 papers.

Sort:

By area:

  1. 2025MR-FlowDPO: Multi-Reward Direct Preference Optimization for Flow-Matching Text-to-Music Generationcited by 5
  2. 2026Stable Audio 3cited by 5
  3. 2025SoundReactor: Frame-level Online Video-to-Audio Generationcited by 4
  4. 2025Melodia: Training-Free Music Editing Guided by Attention Probing in Diffusion Modelscited by 3
  5. 2025Auto-Regressive vs Flow-Matching: a Comparative Study of Modeling Paradigms for Text-to-Music Generationcited by 2
  6. 2026DARC: Drum accompaniment generation with fine-grained rhythm controlcited by 2
  7. 2025Not that Groove: Zero-Shot Symbolic Music Editingcited by 1
  8. 2025Evaluating Music Context Preservation: A Multi-facet Framework for Music Editing Systemscited by 1
  9. 2026Live Music Diffusion Models: Efficient Fine-Tuning and Post-Training of Interactive Diffusion Music Generatorscited by 1
  10. 2025Virtual Consistency for Audio Editingcited by 0
  11. 2025MusRec: Zero-Shot Text-to-Music Editing via Rectified Flow and Diffusion Transformerscited by 0
  12. 2026FlowSonic: Stable Zero-Shot Music Editing via High-Order Trajectory Integrationcited by 0
1 more
  1. 2026Synthesis and editing of multi-instrument audio mixtures using scalar-quantised latents with MIDI Span conditioningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Editing Music with Melody and Text: Using ControlNet for Diffusion Transformer
  2. 2023InstructME: An Instruction Guided Music Edit And Remix Framework with Latent Diffusion Models
  3. 2025RFM-Editing: Rectified Flow Matching for Text-guided Audio Editing
  4. 2026RFM-Editing 2: Text-Guided Audio Editing with Rectified Flow Matching and Coarse-to-Fine Diffusion Transformers
  5. 2025SteerMusic: Enhanced Musical Consistency for Zero-shot Text-guided and Personalized Music Editing
  6. 2024Improving Musical Accompaniment Co-creation via Diffusion Transformers
  7. 2024Quality-aware Masked Diffusion Transformer for Enhanced Music Generation
  8. 2024MEDIC: Zero-shot Music Editing with Disentangled Inversion Control