mlsanityData to 7 Oct 2026

MusicFlow: Cascaded Flow Matching for Text Guided Music Generation

arXiv 2410.20478 ↗First posted 27 Oct 2024cs.SD cs.AI eess.AScited by 14

Not on the map. Why?

Abstract (arXiv)

We introduce MusicFlow, a cascaded text-to-music generation model based on flow matching. Based on self-supervised representations to bridge between text descriptions and music audios, we construct two flow matching networks to model the conditional distribution of semantic and acoustic features. Additionally, we leverage masked prediction as the training objective, enabling the model to generalize to other tasks such as music infilling and continuation in a zero-shot manner. Experiments on MusicCaps reveal that the music generated by MusicFlow exhibits superior quality and text coherence despite being over 2~5 times smaller and requiring 5 times fewer iterative steps. Simultaneously, the model can perform other music generation tasks and achieves competitive performance in music infilling and continuation. Our code and model will be publicly available.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 38 of its 46 references that are in the map. First posted on arXiv 27 Oct 2024.

Builds on

38 of 46 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2022Flow Matching for Generative Modelingcited by 5,426
  8. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  9. 2016Improved Techniques for Training GANscited by 2,938
  10. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  11. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  12. 2021SoundStream: An End-to-End Neural Audio Codeccited by 756
26 more
  1. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  2. 2022High Fidelity Neural Audio Compressioncited by 673
  3. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 592
  4. 2019PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognitioncited by 563
  5. 2023Simple and Controllable Music Generationcited by 480
  6. 2023MusicLM: Generating Music From Textcited by 442
  7. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 423
  8. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 373
  9. 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretrainingcited by 343
  10. 2022AudioGen: Textually Guided Audio Generationcited by 292
  11. 2018Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Datasetcited by 215
  12. 2024Fast Timing-Conditioned Latent Audio Diffusioncited by 178
  13. 2023Noise2Music: Text-conditioned Music Generation with Diffusion Modelscited by 173
  14. 2022Comparative layer-wise analysis of self-supervised speech modelscited by 155
  15. 2023Diff-Foley: Synchronized Video-to-Audio Synthesis with Latent Diffusion Modelscited by 133
  16. 2023UniAudio: An Audio Foundation Model Toward Universal Audio Generationcited by 127
  17. 2023Audiobox: Unified Audio Generation with Natural Language Promptscited by 120
  18. 2023Guided Flows for Generative Modeling and Decision Makingcited by 99
  19. 2023Efficient Neural Music Generationcited by 50
  20. 2024Masked Audio Generation using a Single Non-Autoregressive Transformercited by 50
  21. 2021Sequence-to-Sequence Piano Transcription with Transformerscited by 48
  22. 2023JEN-1: Text-Guided Universal Music Generation with Omnidirectional Diffusion Modelscited by 41
  23. 2023Bespoke Solvers for Generative Flow Modelscited by 36
  24. 2022A Lightweight Instrument-Agnostic Model for Polyphonic Note Transcription and Multipitch Estimationcited by 27
  25. 2019Improving Automatic Jazz Melody Generation by Transfer Learning Techniquescited by 8
  26. 2023Self-Supervised Music Source Separation Using Vector-Quantized Source Category Estimatescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 6.

14 papers.

Sort:

By area:

  1. 2025FLowHigh: Towards Efficient and High-Quality Audio Super-Resolution with Single-Step Flow Matchingcited by 18
  2. 2025Audio Super-Resolution with Latent Bridge Modelscited by 13
  3. 2025InspireMusic: Integrating Super Resolution and Large Language Model for High-Fidelity Long-Form Music Generationcited by 10
  4. 2025From Faces to Voices: Learning Hierarchical Representations for High-quality Video-to-Speechcited by 9
  5. 2025MR-FlowDPO: Multi-Reward Direct Preference Optimization for Flow-Matching Text-to-Music Generationcited by 5
  6. 2025MusFlow: Multimodal Music Generation via Conditional Flow Matchingcited by 4
  7. 2025Is There a Better Source Distribution than Gaussian? Exploring Source Distributions for Image Flow Matchingcited by 4
  8. 2025FlowSynth: Instrument Generation Through Distributional Flow Matching and Test-Time Searchcited by 2
  9. 2026Geometry-Aware Image Flow Matchingcited by 2
  10. 2026Anysynth:Zero-Shot Instrument Cloning via In-Context Learning and Asymmetric Hierarchical Guidancecited by 1
  11. 2026Training-Efficient Text-to-Music Generation with State-Space Modelingcited by 0
  12. 2026S2Accompanist: A Semantic-Aware and Structure-Guided Diffusion Model for Music Accompaniment Generationcited by 0
2 more
  1. 2026JenBridge: Adaptive Long-Form Video Soundtracking across Scene Transitionscited by 0
  2. 2026SketchSong: Hierarchical Song Generation with Sketch Planning and Fine-Grained Multi-Track Modelingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Auto-Regressive vs Flow-Matching: a Comparative Study of Modeling Paradigms for Text-to-Music Generation
  2. 2026Pushing the Frontier of Full-Song Generation: Hierarchical Autoregressive Planning Meets Flow-Matching Rendering
  3. 2024FLUX that Plays Music
  4. 2023Mousai: Text-to-Music Generation with Long-Context Latent Diffusion
  5. 2024Quality-aware Masked Diffusion Transformer for Enhanced Music Generation
  6. 2025JAM: A Tiny Flow-based Song Generator with Fine-grained Controllability and Aesthetic Alignment
  7. 2023JEN-1 Composer: A Unified Framework for High-Fidelity Multi-Track Music Generation
  8. 2024High Fidelity Text-Guided Music Editing via Single-Stage Flow Matching