# YuE: Scaling Open Foundation Models for Long-Form Music Generation

arXiv: https://arxiv.org/abs/2503.08638 · HTML: https://mlsanity.com/paper/2503.08638/

## Key facts

- Cited by 71 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Mar 2025.

## Built on by (most cited first, 30 of 71)

- [OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMs](https://mlsanity.com/paper/2510.10689/) (2025)
- [ACE-Step: A Step Towards Music Generation Foundation Model](https://mlsanity.com/paper/2506.00045/) (2025)
- [LeVo: High-Quality Song Generation with Multi-Preference Alignment](https://mlsanity.com/paper/2506.07520/) (2025)
- [SongBloom: Coherent Song Generation via Interleaved Autoregressive Sketching and Diffusion Refinement](https://mlsanity.com/paper/2506.07634/) (2025)
- [Large Language Model Reasoning Failures](https://mlsanity.com/paper/2602.06176/) (2026)
- [DiffRhythm 2: Efficient and High Fidelity Song Generation via Block Flow Matching](https://mlsanity.com/paper/2510.22950/) (2025)
- [Analyzable Chain-of-Musical-Thought Prompting for High-Fidelity Music Generation](https://mlsanity.com/paper/2503.19611/) (2025)
- [Audio Super-Resolution with Latent Bridge Models](https://mlsanity.com/paper/2509.17609/) (2025)
- [Live Music Models](https://mlsanity.com/paper/2508.04651/) (2025)
- [UALM: Unified Audio Language Model for Understanding, Generation and Reasoning](https://mlsanity.com/paper/2510.12000/) (2025)
- [AutoMV: An Automatic Multi-Agent System for Music Video Generation](https://mlsanity.com/paper/2512.12196/) (2025)
- [Muse: Towards Reproducible Long-Form Song Generation with Fine-Grained Style Control](https://mlsanity.com/paper/2601.03973/) (2026)
- [SingMOS-Pro: An Comprehensive Benchmark for Singing Quality Assessment](https://mlsanity.com/paper/2510.01812/) (2025)
- [SongBench: A Fine-Grained Multi-Aspect Benchmark for Song Quality Assessment](https://mlsanity.com/paper/2604.25937/) (2026)
- [MMMG: a Comprehensive and Reliable Benchmark for Multitask Multimodal Generation](https://mlsanity.com/paper/2505.17613/) (2025)
- [Steering Autoregressive Music Generation with Recursive Feature Machines](https://mlsanity.com/paper/2510.19127/) (2025)
- [Unison: Harmonizing Motion, Speech, and Sound for Human-Centric Audio-Video Generation](https://mlsanity.com/paper/2605.08729/) (2026)
- [Stable Audio 3](https://mlsanity.com/paper/2605.17991/) (2026)
- [LeVo 2: Stable and Melodious Song Generation via Hierarchical Representation Modeling and Progressive Post-Training](https://mlsanity.com/paper/2606.30642/) (2026)
- [Vevo2: A Unified and Controllable Framework for Speech and Singing Voice Generation](https://mlsanity.com/paper/2508.16332/) (2025)
- [CMI-RewardBench: Evaluating Music Reward Models with Compositional Multimodal Instruction](https://mlsanity.com/paper/2603.00610/) (2026)
- [Vision-to-Music Generation: A Survey](https://mlsanity.com/paper/2503.21254/) (2025)
- [Automatic Stage Lighting Control: Is it a Rule-Driven Process or Generative Task?](https://mlsanity.com/paper/2506.01482/) (2025)
- [UniMoE-Audio: Unified Speech and Music Generation with Dynamic-Capacity MoE](https://mlsanity.com/paper/2510.13344/) (2025)
- [Bagpiper: Solving Open-Ended Audio Tasks via Rich Captions](https://mlsanity.com/paper/2602.05220/) (2026)
- [Pushing the Frontier of Full-Song Generation: Hierarchical Autoregressive Planning Meets Flow-Matching Rendering](https://mlsanity.com/paper/2607.20253/) (2026)
- [Bob's Confetti: Phonetic Memorization Attacks in Music and Video Generation](https://mlsanity.com/paper/2507.17937/) (2025)
- [Via Score to Performance: Efficient Human-Controllable Long Song Generation with Bar-Level Symbolic Notation](https://mlsanity.com/paper/2508.01394/) (2025)
- [SupCLAP: Controlling Optimization Trajectory Drift in Audio-Text Contrastive Learning with Support Vector Regularization](https://mlsanity.com/paper/2509.21033/) (2025)
- [SoulX-Singer: Towards High-Quality Zero-Shot Singing Voice Synthesis](https://mlsanity.com/paper/2602.07803/) (2026)

## Abstract (arXiv)

We tackle the task of long-form music generation--particularly the challenging lyrics-to-song problem--by introducing YuE, a family of open foundation models based on the LLaMA2 architecture. Specifically, YuE scales to trillions of tokens and generates up to five minutes of music while maintaining lyrical alignment, coherent musical structure, and engaging vocal melodies with appropriate accompaniment. It achieves this through (1) track-decoupled next-token prediction to overcome dense mixture signals, (2) structural progressive conditioning for long-context lyrical alignment, and (3) a multitask, multiphase pre-training recipe to converge and generalize. In addition, we redesign the in-context learning technique for music generation, enabling versatile style transfer (e.g., converting Japanese city pop into an English rap while preserving the original accompaniment) and bidirectional generation. Through extensive evaluation, we demonstrate that YuE matches or even surpasses some of the proprietary systems in musicality and vocal agility. In addition, fine-tuning YuE enables additional controls and enhanced support for tail languages. Furthermore, beyond generation, we show that YuE's learned representations can perform well on music understanding tasks, where the results of YuE match or exceed state-of-the-art methods on the MARBLE benchmark. Keywords: lyrics2song, song generation, long-form, foundation model, music generation