# Encoding Musical Style with Transformer Autoencoders

arXiv: https://arxiv.org/abs/1912.05537 · HTML: https://mlsanity.com/paper/1912.05537/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Dec 2019.

## Built on by (most cited first, 30 of 35)

- [Symbolic Music Generation with Diffusion Models](https://mlsanity.com/paper/2103.16091/) (2021)
- [Compound Word Transformer: Learning to Compose Full-Song Music over Dynamic Directed Hypergraphs](https://mlsanity.com/paper/2101.02402/) (2021)
- [Symbolic Music Generation with Non-Differentiable Rule Guided Diffusion](https://mlsanity.com/paper/2402.14285/) (2024)
- [MuseCoco: Generating Symbolic Music from Text](https://mlsanity.com/paper/2306.00110/) (2023)
- [MuseMorphose: Full-Song and Fine-Grained Piano Music Style Transfer with One Transformer VAE](https://mlsanity.com/paper/2105.04090/) (2021)
- [Attention Mechanism in Neural Networks: Where it Comes and Where it Goes](https://mlsanity.com/paper/2204.13154/) (2022)
- [Byte Pair Encoding for Symbolic Music](https://mlsanity.com/paper/2301.11975/) (2023)
- [V2Meow: Meowing to the Visual Beat via Video-to-Music Generation](https://mlsanity.com/paper/2305.06594/) (2023)
- [Variable-Length Music Score Infilling via XLNet and Musically Specialized Positional Encoding](https://mlsanity.com/paper/2108.05064/) (2021)
- [Symbolic music generation conditioned on continuous-valued emotions](https://mlsanity.com/paper/2203.16165/) (2022)
- [PART: Pre-trained Authorship Representation Transformer](https://mlsanity.com/paper/2209.15373/) (2022)
- [Transformers in Reinforcement Learning: A Survey](https://mlsanity.com/paper/2307.05979/) (2023)
- [Understanding writing style in social media with a supervised contrastively pre-trained transformer](https://mlsanity.com/paper/2310.11081/) (2023)
- [A Review of Intelligent Music Generation Systems](https://mlsanity.com/paper/2211.09124/) (2022)
- [Pop2Piano : Pop Audio-based Piano Cover Generation](https://mlsanity.com/paper/2211.00895/) (2022)
- [Composer Style-specific Symbolic Music Generation Using Vector Quantized Discrete Diffusion Models](https://mlsanity.com/paper/2310.14044/) (2023)
- [Ember: No-Code Context Enrichment via Similarity-Based Keyless Joins](https://mlsanity.com/paper/2106.01501/) (2021)
- [Reconstructing Human Expressiveness in Piano Performances with a Transformer Network](https://mlsanity.com/paper/2306.06040/) (2023)
- [Generative Pretraining for Paraphrase Evaluation](https://mlsanity.com/paper/2107.08251/) (2021)
- [Mamba-Diffusion Model with Learnable Wavelet for Controllable Symbolic Music Generation](https://mlsanity.com/paper/2505.03314/) (2025)
- [Unifying Symbolic Music Arrangement: Track-Aware Reconstruction and Structured Tokenization](https://mlsanity.com/paper/2408.15176/) (2024)
- [A Contextual Latent Space Model: Subsequence Modulation in Melodic Sequence](https://mlsanity.com/paper/2111.11703/) (2021)
- [A Survey of Music Generation in the Context of Interaction](https://mlsanity.com/paper/2402.15294/) (2024)
- [BandCondiNet: Parallel Transformers-based Conditional Popular Music Generation with Multi-View Features](https://mlsanity.com/paper/2407.10462/) (2024)
- [WaveTransfer: A Flexible End-to-end Multi-instrument Timbre Transfer with Diffusion](https://mlsanity.com/paper/2409.15321/) (2024)
- [Calliope -- A Polyphonic Music Transformer](https://mlsanity.com/paper/2107.05546/) (2021)
- [FIGARO: Generating Symbolic Music with Fine-Grained Artistic Control](https://mlsanity.com/paper/2201.10936/) (2022)
- [Efficient Fine-Grained Guidance for Diffusion Model Based Symbolic Music Generation](https://mlsanity.com/paper/2410.08435/) (2024)
- [LZMidi: Compression-Based Symbolic Music Generation](https://mlsanity.com/paper/2503.17654/) (2025)
- [MuSLCAT: Multi-Scale Multi-Level Convolutional Attention Transformer for Discriminative Music Modeling on Raw Waveforms](https://mlsanity.com/paper/2104.02309/) (2021)

## Abstract (arXiv)

We consider the problem of learning high-level controls over the global structure of generated sequences, particularly in the context of symbolic music generation with complex language models. In this work, we present the Transformer autoencoder, which aggregates encodings of the input data across time to obtain a global representation of style from a given performance. We show it is possible to combine this global representation with other temporally distributed embeddings, enabling improved control over the separate aspects of performance style and melody. Empirically, we demonstrate the effectiveness of our method on various music generation tasks on the MAESTRO dataset and a YouTube dataset with 10,000+ hours of piano performances, where we achieve improvements in terms of log-likelihood and mean listening scores as compared to baselines.