# High Fidelity Text-Guided Music Editing via Single-Stage Flow Matching

arXiv: https://arxiv.org/abs/2407.03648 · HTML: https://mlsanity.com/paper/2407.03648/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 46 of its 52 references that are in the map.
- First posted on arXiv 4 Jul 2024.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 30 of 46)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022)
- [Improving and generalizing flow-based generative models with minibatch optimal transport](https://mlsanity.com/paper/2302.00482/) (2023)
- [SoundStream: An End-to-End Neural Audio Codec](https://mlsanity.com/paper/2107.03312/) (2021)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel](https://mlsanity.com/paper/2304.11277/) (2023)
- [AudioLDM: Text-to-Audio Generation with Latent Diffusion Models](https://mlsanity.com/paper/2301.12503/) (2023)
- [Zero-shot Image-to-Image Translation](https://mlsanity.com/paper/2302.03027/) (2023)
- [All are Worth Words: A ViT Backbone for Diffusion Models](https://mlsanity.com/paper/2209.12152/) (2022)
- [Simple and Controllable Music Generation](https://mlsanity.com/paper/2306.05284/) (2023)
- [MusicLM: Generating Music From Text](https://mlsanity.com/paper/2301.11325/) (2023)
- [High-Fidelity Audio Compression with Improved RVQGAN](https://mlsanity.com/paper/2306.06546/) (2023)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale](https://mlsanity.com/paper/2306.15687/) (2023)
- [AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining](https://mlsanity.com/paper/2308.05734/) (2023)
- [Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation](https://mlsanity.com/paper/2303.00848/) (2023)
- [AudioGen: Textually Guided Audio Generation](https://mlsanity.com/paper/2209.15352/) (2022)
- [Multisample Flow Matching: Straightening Flows with Minibatch Couplings](https://mlsanity.com/paper/2304.14772/) (2023)
- [Masked Autoencoders that Listen](https://mlsanity.com/paper/2207.06405/) (2022)
- [An Edit Friendly DDPM Noise Space: Inversion and Manipulations](https://mlsanity.com/paper/2304.06140/) (2023)
- [Efficient Training of Audio Transformers with Patchout](https://mlsanity.com/paper/2110.05069/) (2021)
- [Fast Timing-Conditioned Latent Audio Diffusion](https://mlsanity.com/paper/2402.04825/) (2024)
- [Noise2Music: Text-conditioned Music Generation with Diffusion Models](https://mlsanity.com/paper/2302.03917/) (2023)
- [Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model](https://mlsanity.com/paper/2304.13731/) (2023)

## Built on by (most cited first, 13 of 13)

- [Stable Audio 3](https://mlsanity.com/paper/2605.17991/) (2026)
- [SoundReactor: Frame-level Online Video-to-Audio Generation](https://mlsanity.com/paper/2510.02110/) (2025)
- [MR-FlowDPO: Multi-Reward Direct Preference Optimization for Flow-Matching Text-to-Music Generation](https://mlsanity.com/paper/2512.10264/) (2025)
- [Melodia: Training-Free Music Editing Guided by Attention Probing in Diffusion Models](https://mlsanity.com/paper/2511.08252/) (2025)
- [Auto-Regressive vs Flow-Matching: a Comparative Study of Modeling Paradigms for Text-to-Music Generation](https://mlsanity.com/paper/2506.08570/) (2025)
- [DARC: Drum accompaniment generation with fine-grained rhythm control](https://mlsanity.com/paper/2601.02357/) (2026)
- [Not that Groove: Zero-Shot Symbolic Music Editing](https://mlsanity.com/paper/2505.08203/) (2025)
- [Evaluating Music Context Preservation: A Multi-facet Framework for Music Editing Systems](https://mlsanity.com/paper/2512.14629/) (2025)
- [Live Music Diffusion Models: Efficient Fine-Tuning and Post-Training of Interactive Diffusion Music Generators](https://mlsanity.com/paper/2605.22717/) (2026)
- [Virtual Consistency for Audio Editing](https://mlsanity.com/paper/2509.17219/) (2025)
- [MusRec: Zero-Shot Text-to-Music Editing via Rectified Flow and Diffusion Transformers](https://mlsanity.com/paper/2511.04376/) (2025)
- [FlowSonic: Stable Zero-Shot Music Editing via High-Order Trajectory Integration](https://mlsanity.com/paper/2607.17526/) (2026)
- [Synthesis and editing of multi-instrument audio mixtures using scalar-quantised latents with MIDI Span conditioning](https://mlsanity.com/paper/2609.25546/) (2026)

## Abstract (arXiv)

We introduce MelodyFlow, an efficient text-controllable high-fidelity music generation and editing model. It operates on continuous latent representations from a low frame rate 48 kHz stereo variational auto encoder codec. Based on a diffusion transformer architecture trained on a flow-matching objective the model can edit diverse high quality stereo samples of variable duration, with simple text descriptions. We adapt the ReNoise latent inversion method to flow matching and compare it with the original implementation and naive denoising diffusion implicit model (DDIM) inversion on a variety of music editing prompts. Our results indicate that our latent inversion outperforms both ReNoise and DDIM for zero-shot test-time text-guided editing on several objective metrics. Subjective evaluations exhibit a substantial improvement over previous state of the art for music editing. Code and model weights will be publicly made available. Samples are available at https://melodyflow.github.io.