# Compose & Embellish: Well-Structured Piano Performance Generation via A Two-Stage Approach

arXiv: https://arxiv.org/abs/2209.08212 · HTML: https://mlsanity.com/paper/2209.08212/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 24 references that are in the map.
- First posted on arXiv 17 Sep 2022.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [Music Transformer](https://mlsanity.com/paper/1809.04281/) (2018)
- [Compound Word Transformer: Learning to Compose Full-Song Music over Dynamic Directed Hypergraphs](https://mlsanity.com/paper/2101.02402/) (2021)
- [A Theoretical Analysis of the Repetition Problem in Text Generation](https://mlsanity.com/paper/2012.14660/) (2020)
- [The Jazz Transformer on the Front Line: Exploring the Shortcomings of AI-composed Music through Quantitative Measures](https://mlsanity.com/paper/2008.01307/) (2020)
- [MuseMorphose: Full-Song and Fine-Grained Piano Music Style Transfer with One Transformer VAE](https://mlsanity.com/paper/2105.04090/) (2021)
- [Theme Transformer: Symbolic Music Generation with Theme-Conditioned Transformer](https://mlsanity.com/paper/2111.04093/) (2021)
- [Controllable deep melody generation via hierarchical music structure representation](https://mlsanity.com/paper/2109.00663/) (2021)
- [Automatic Analysis and Influence of Hierarchical Structure on Melody, Rhythm and Harmony in Popular Music](https://mlsanity.com/paper/2010.07518/) (2020)
- [MELONS: generating melody with long-term structure using transformers and structure graph](https://mlsanity.com/paper/2110.05020/) (2021)
- [Pop Music Transformer: Beat-based Modeling and Generation of Expressive Pop Piano Compositions](https://mlsanity.com/paper/2002.00212/) (2020)
- [BERT-like Pre-training for Symbolic Piano Music Classification Tasks](https://mlsanity.com/paper/2107.05223/) (2021)
- [Structure-Enhanced Pop Music Generation via Harmony-Aware Learning](https://mlsanity.com/paper/2109.06441/) (2021)
- [What is missing in deep music generation? A study of repetition and structure in popular music](https://mlsanity.com/paper/2209.00182/) (2022)

## Built on by (most cited first, 18 of 18)

- [PDMX: A Large-Scale Public Domain MusicXML Dataset for Symbolic Music Processing](https://mlsanity.com/paper/2409.10831/) (2024)
- [MelodyGLM: Multi-task Pre-training for Symbolic Melody Generation](https://mlsanity.com/paper/2309.10738/) (2023)
- [Natural Language Processing Methods for Symbolic Music Generation and Information Retrieval: a Survey](https://mlsanity.com/paper/2402.17467/) (2024)
- [Motifs, Phrases, and Beyond: The Modelling of Structure in Symbolic Music Generation](https://mlsanity.com/paper/2403.07995/) (2024)
- [Emotion-driven Piano Music Generation via Two-stage Disentanglement and Functional Representation](https://mlsanity.com/paper/2407.20955/) (2024)
- [Emotion-Driven Melody Harmonization via Melodic Variation and Functional Representation](https://mlsanity.com/paper/2407.20176/) (2024)
- [PerTok: Expressive Encoding and Modeling of Symbolic Musical Ideas and Variations](https://mlsanity.com/paper/2410.02060/) (2024)
- [Hookpad Aria: A Copilot for Songwriters](https://mlsanity.com/paper/2502.08122/) (2025)
- [From Words to Music: A Study of Subword Tokenization Techniques in Symbolic Music Generation](https://mlsanity.com/paper/2304.08953/) (2023)
- [Time-Shifted Token Scheduling for Symbolic Music Generation](https://mlsanity.com/paper/2509.23749/) (2025)
- [Segment-Factorized Full-Song Generation on Symbolic Piano Music](https://mlsanity.com/paper/2510.05881/) (2025)
- [Music Proofreading with RefinPaint: Where and How to Modify Compositions given Context](https://mlsanity.com/paper/2407.09099/) (2024)
- [From Generality to Mastery: Composer-Style Symbolic Music Generation via Large-Scale Pre-training](https://mlsanity.com/paper/2506.17497/) (2025)
- [Etude: Piano Cover Generation with a Three-Stage Approach -- Extract, strucTUralize, and DEcode](https://mlsanity.com/paper/2509.16522/) (2025)
- [MIDI-LLM: Improving Text-to-MIDI Music Generation via Adapting Large Language Models](https://mlsanity.com/paper/2511.03942/) (2025)
- [Unsupervised Lead Sheet Generation via Semantic Compression](https://mlsanity.com/paper/2310.10772/) (2023)
- [D3PIA: A Discrete Denoising Diffusion Model for Piano Accompaniment Generation From Lead sheet](https://mlsanity.com/paper/2602.03523/) (2026)
- [Harmonic Representations in Symbolic Music Transformers are Localized and Controllable](https://mlsanity.com/paper/2610.15957/) (2026)

## Abstract (arXiv)

Even with strong sequence models like Transformers, generating expressive piano performances with long-range musical structures remains challenging. Meanwhile, methods to compose well-structured melodies or lead sheets (melody + chords), i.e., simpler forms of music, gained more success. Observing the above, we devise a two-stage Transformer-based framework that Composes a lead sheet first, and then Embellishes it with accompaniment and expressive touches. Such a factorization also enables pretraining on non-piano data. Our objective and subjective experiments show that Compose & Embellish shrinks the gap in structureness between a current state of the art and real performances by half, and improves other musical aspects such as richness and coherence as well.