# SongBloom: Coherent Song Generation via Interleaved Autoregressive Sketching and Diffusion Refinement

arXiv: https://arxiv.org/abs/2506.07634 · HTML: https://mlsanity.com/paper/2506.07634/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 40 of its 49 references that are in the map.
- First posted on arXiv 9 Jun 2025.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 30 of 40)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [ZeRO: Memory Optimizations Toward Training Trillion Parameter Models](https://mlsanity.com/paper/1910.02054/) (2019)
- [SoundStream: An End-to-End Neural Audio Codec](https://mlsanity.com/paper/2107.03312/) (2021)
- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://mlsanity.com/paper/2301.02111/) (2023)
- [Autoregressive Image Generation without Vector Quantization](https://mlsanity.com/paper/2406.11838/) (2024)
- [Jukebox: A Generative Model for Music](https://mlsanity.com/paper/2005.00341/) (2020)
- [Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model](https://mlsanity.com/paper/2408.11039/) (2024)
- [MusicLM: Generating Music From Text](https://mlsanity.com/paper/2301.11325/) (2023)
- [WhisperX: Time-Accurate Speech Transcription of Long-Form Audio](https://mlsanity.com/paper/2303.00747/) (2023)
- [Stable Audio Open](https://mlsanity.com/paper/2407.14358/) (2024)
- [MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training](https://mlsanity.com/paper/2306.00107/) (2023)
- [Meta Audiobox Aesthetics: Unified Automatic Quality Assessment for Speech, Music, and Sound](https://mlsanity.com/paper/2502.05139/) (2025)
- [Hybrid Transformers for Music Source Separation](https://mlsanity.com/paper/2211.08553/) (2022)
- [AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation](https://mlsanity.com/paper/2305.09515/) (2023)
- [MuLan: A Joint Embedding of Music Audio and Natural Language](https://mlsanity.com/paper/2208.12415/) (2022)
- [YuE: Scaling Open Foundation Models for Long-Form Music Generation](https://mlsanity.com/paper/2503.08638/) (2025)
- [MuQ: Self-Supervised Music Representation Learning with Mel Residual Vector Quantization](https://mlsanity.com/paper/2501.01108/) (2025)
- [DiTAR: Diffusion Transformer Autoregressive Modeling for Speech Generation](https://mlsanity.com/paper/2502.03930/) (2025)
- [ACE-Step: A Step Towards Music Generation Foundation Model](https://mlsanity.com/paper/2506.00045/) (2025)
- [Efficient Neural Music Generation](https://mlsanity.com/paper/2305.15719/) (2023)
- [Multimodal Latent Language Modeling with Next-Token Diffusion](https://mlsanity.com/paper/2412.08635/) (2024)
- [DiffRhythm: Blazingly Fast and Embarrassingly Simple End-to-End Full-Length Song Generation with Latent Diffusion](https://mlsanity.com/paper/2503.01183/) (2025)

## Built on by (most cited first, 23 of 23)

- [Self-Supervised Flow Matching for Scalable Multi-Modal Synthesis](https://mlsanity.com/paper/2603.06507/) (2026)
- [DiffRhythm 2: Efficient and High Fidelity Song Generation via Block Flow Matching](https://mlsanity.com/paper/2510.22950/) (2025)
- [Muse: Towards Reproducible Long-Form Song Generation with Fine-Grained Style Control](https://mlsanity.com/paper/2601.03973/) (2026)
- [SongBench: A Fine-Grained Multi-Aspect Benchmark for Song Quality Assessment](https://mlsanity.com/paper/2604.25937/) (2026)
- [LeVo 2: Stable and Melodious Song Generation via Hierarchical Representation Modeling and Progressive Post-Training](https://mlsanity.com/paper/2606.30642/) (2026)
- [SongPrep: A Preprocessing Framework and End-to-end Model for Full-song Structure Parsing and Lyrics Transcription](https://mlsanity.com/paper/2509.17404/) (2025)
- [Direct Preference Optimization for Speech Autoregressive Diffusion Models](https://mlsanity.com/paper/2509.18928/) (2025)
- [Pushing the Frontier of Full-Song Generation: Hierarchical Autoregressive Planning Meets Flow-Matching Rendering](https://mlsanity.com/paper/2607.20253/) (2026)
- [SoulX-Singer: Towards High-Quality Zero-Shot Singing Voice Synthesis](https://mlsanity.com/paper/2602.07803/) (2026)
- [Video-Robin: Autoregressive Diffusion Planning for Intent-Grounded Video-to-Music Generation](https://mlsanity.com/paper/2604.17656/) (2026)
- [Shao: Scaling Acoustic Token Language Models Toward High-Fidelity Music Generation](https://mlsanity.com/paper/2605.01790/) (2026)
- [Anysynth:Zero-Shot Instrument Cloning via In-Context Learning and Asymmetric Hierarchical Guidance](https://mlsanity.com/paper/2607.11143/) (2026)
- [Can LLM design high-quality experiments? A Comprehensive and Systematic Benchmark on Autonomous Experimental Design](https://mlsanity.com/paper/2608.03501/) (2026)
- [Tutti: Expressive Multi-Singer Synthesis via Structure-Level Timbre Control and Vocal Texture Modeling](https://mlsanity.com/paper/2602.08233/) (2026)
- [LaDA-Band: Language Diffusion Models for Vocal-to-Accompaniment Generation](https://mlsanity.com/paper/2604.11052/) (2026)
- [SketchSong: Hierarchical Song Generation with Sketch Planning and Fine-Grained Multi-Track Modeling](https://mlsanity.com/paper/2606.03169/) (2026)
- [UniVoice: A Unified Model for Speech and Singing Voice Generation](https://mlsanity.com/paper/2606.05852/) (2026)
- [MPEcho: A Melody and Phoneme-Aware Generative Framework for Controllable Cover Song Generation](https://mlsanity.com/paper/2607.26698/) (2026)
- [VIBE: Video Instruction-aligned Background music gEneration](https://mlsanity.com/paper/2608.30125/) (2026)
- [DiffSynth-Music: Audio-Conditioned KV-Cache Adapters for Controllable Music Generation](https://mlsanity.com/paper/2609.12774/) (2026)
- [Open-Qwen-Music: An Auditable Framework for LLM-Based Music Composition and Diffusion Rendering](https://mlsanity.com/paper/2609.31652/) (2026)
- [YuE2: Unifying Symbolic and Audio Music Generation at Frontier Quality](https://mlsanity.com/paper/2609.33757/) (2026)
- [Ghost in the Encoder: Decodable Artist Identity Representations in Lyrics-to-Song Generation](https://mlsanity.com/paper/2609.39552/) (2026)

## Abstract (arXiv)

Generating music with coherent structure, harmonious instrumental and vocal elements remains a significant challenge in song generation. Existing language models and diffusion-based methods often struggle to balance global coherence with local fidelity, resulting in outputs that lack musicality or suffer from incoherent progression and mismatched lyrics. This paper introduces SongBloom, a novel framework for full-length song generation that leverages an interleaved paradigm of autoregressive sketching and diffusion-based refinement. SongBloom employs an autoregressive diffusion model that combines the high fidelity of diffusion models with the scalability of language models. Specifically, it gradually extends a musical sketch from short to long and refines the details from coarse to fine-grained. The interleaved generation paradigm effectively integrates prior semantic and acoustic context to guide the generation process. Experimental results demonstrate that SongBloom outperforms existing methods across both subjective and objective metrics and achieves performance comparable to the state-of-the-art commercial music generation platforms. Audio samples are available on our demo page: https://cypress-yang.github.io/SongBloomdemo. The code and model weights have been released on https://github.com/Cypress-Yang/SongBloom .