# Variable-Length Music Score Infilling via XLNet and Musically Specialized Positional Encoding

arXiv: https://arxiv.org/abs/2108.05064 · HTML: https://mlsanity.com/paper/2108.05064/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 41 references that are in the map.
- First posted on arXiv 11 Aug 2021.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [Music Transformer](https://mlsanity.com/paper/1809.04281/) (2018)
- [A Comparative Study on Transformer vs RNN in Speech Applications](https://mlsanity.com/paper/1909.06317/) (2019)
- [Rethinking Positional Encoding in Language Pre-training](https://mlsanity.com/paper/2006.15595/) (2020)
- [Enabling Language Models to Fill in the Blanks](https://mlsanity.com/paper/2005.05339/) (2020)
- [Compound Word Transformer: Learning to Compose Full-Song Music over Dynamic Directed Hypergraphs](https://mlsanity.com/paper/2101.02402/) (2021)
- [DeepBach: a Steerable Model for Bach Chorales Generation](https://mlsanity.com/paper/1612.01010/) (2016)
- [MaskGAN: Better Text Generation via Filling in the______](https://mlsanity.com/paper/1801.07736/) (2018)
- [This Time with Feeling: Learning Expressive Musical Performance](https://mlsanity.com/paper/1808.03715/) (2018)
- [PopMAG: Pop Music Accompaniment Generation](https://mlsanity.com/paper/2008.07703/) (2020)
- [Felix: Flexible Text Editing Through Tagging and Insertion](https://mlsanity.com/paper/2003.10687/) (2020)
- [LakhNES: Improving multi-instrumental music generation with cross-domain pre-training](https://mlsanity.com/paper/1907.04868/) (2019)
- [The Jazz Transformer on the Front Line: Exploring the Shortcomings of AI-composed Music through Quantitative Measures](https://mlsanity.com/paper/2008.01307/) (2020)
- [Counterpoint by Convolution](https://mlsanity.com/paper/1903.07227/) (2019)
- [Blank Language Models](https://mlsanity.com/paper/2002.03079/) (2020)
- [MuseMorphose: Full-Song and Fine-Grained Piano Music Style Transfer with One Transformer VAE](https://mlsanity.com/paper/2105.04090/) (2021)
- [Relative Positional Encoding for Transformers with Linear Complexity](https://mlsanity.com/paper/2105.08399/) (2021)
- [Music transcription modelling and composition using deep learning](https://mlsanity.com/paper/1604.08723/) (2016)
- [Encoding Musical Style with Transformer Autoencoders](https://mlsanity.com/paper/1912.05537/) (2019)
- [Text Infilling](https://mlsanity.com/paper/1901.00158/) (2019)
- [Music SketchNet: Controllable Music Generation via Factorized Representations of Pitch and Rhythm](https://mlsanity.com/paper/2008.01291/) (2020)
- [Learning to Traverse Latent Spaces for Musical Score Inpainting](https://mlsanity.com/paper/1907.01164/) (2019)
- [Automatic Composition of Guitar Tabs by Transformers and Groove Modeling](https://mlsanity.com/paper/2008.01431/) (2020)
- [NONOTO: A Model-agnostic Web Interface for Interactive Music Composition by Inpainting](https://mlsanity.com/paper/1907.10380/) (2019)
- [Generating Music with a Self-Correcting Non-Chronological Autoregressive Model](https://mlsanity.com/paper/2008.08927/) (2020)

## Built on by (most cited first, 14 of 14)

- [Anticipatory Music Transformer](https://mlsanity.com/paper/2306.08620/) (2023)
- [Polyffusion: A Diffusion Model for Polyphonic Score Generation with Internal and External Controls](https://mlsanity.com/paper/2307.10304/) (2023)
- [Multitrack Music Transformer](https://mlsanity.com/paper/2207.06983/) (2022)
- [Arrange, Inpaint, and Refine: Steerable Long-term Music Audio Generation and Editing via Content-based Controls](https://mlsanity.com/paper/2402.09508/) (2024)
- [Natural Language Processing Methods for Symbolic Music Generation and Information Retrieval: a Survey](https://mlsanity.com/paper/2402.17467/) (2024)
- [Composer's Assistant: An Interactive Transformer for Multi-Track MIDI Infilling](https://mlsanity.com/paper/2301.12525/) (2023)
- [Composer's Assistant 2: Interactive Multi-Track MIDI Infilling with Fine-Grained User Control](https://mlsanity.com/paper/2407.14700/) (2024)
- [Melody Infilling with User-Provided Structural Context](https://mlsanity.com/paper/2210.02829/) (2022)
- [ImprovNet -- Generating Controllable Musical Improvisations with Iterative Corruption Refinement](https://mlsanity.com/paper/2502.04522/) (2025)
- [Music Score Expansion with Variable-Length Infilling](https://mlsanity.com/paper/2111.06046/) (2021)
- [Time-Shifted Token Scheduling for Symbolic Music Generation](https://mlsanity.com/paper/2509.23749/) (2025)
- [BeatEdit: Symbolic Music Generation as Explicit Editing](https://mlsanity.com/paper/2607.11124/) (2026)
- [Exploring Softly Masked Language Modelling for Controllable Symbolic Music Generation](https://mlsanity.com/paper/2305.03530/) (2023)
- [Pictures Of MIDI: Controlled Music Generation via Graphical Prompts for Image-Based Diffusion Inpainting](https://mlsanity.com/paper/2407.01499/) (2024)

## Abstract (arXiv)

This paper proposes a new self-attention based model for music score infilling, i.e., to generate a polyphonic music sequence that fills in the gap between given past and future contexts. While existing approaches can only fill in a short segment with a fixed number of notes, or a fixed time span between the past and future contexts, our model can infill a variable number of notes (up to 128) for different time spans. We achieve so with three major technical contributions. First, we adapt XLNet, an autoregressive model originally proposed for unsupervised model pre-training, to music score infilling. Second, we propose a new, musically specialized positional encoding called relative bar encoding that better informs the model of notes' position within the past and future context. Third, to capitalize relative bar encoding, we perform look-ahead onset prediction to predict the onset of a note one time step before predicting the other attributes of the note. We compare our proposed model with two strong baselines and show that our model is superior in both objective and subjective analyses.