# A variational autoencoder for music generation controlled by tonal tension

arXiv: https://arxiv.org/abs/2010.06230 · HTML: https://mlsanity.com/paper/2010.06230/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2020.

## Built on by (most cited first, 8 of 8)

- [Video2Music: Suitable Music Generation from Videos using an Affective Multimodal Transformer model](https://mlsanity.com/paper/2311.00968/) (2023)
- [Text2midi: Generating Symbolic Music from Captions](https://mlsanity.com/paper/2412.16526/) (2024)
- [MusIAC: An extensible generative framework for Music Infilling Applications with multi-level Control](https://mlsanity.com/paper/2202.05528/) (2022)
- [Symbolic music generation conditioned on continuous-valued emotions](https://mlsanity.com/paper/2203.16165/) (2022)
- [Re-creation of Creations: A New Paradigm for Lyric-to-Melody Generation](https://mlsanity.com/paper/2208.05697/) (2022)
- [Fast Diffusion GAN Model for Symbolic Music Generation Controlled by Emotions](https://mlsanity.com/paper/2310.14040/) (2023)
- [MIDI-Draw: Sketching to Control Melody Generation](https://mlsanity.com/paper/2305.11605/) (2023)
- [Explicit Tonal Tension Conditioning via Dual-Level Beam Search for Symbolic Music Generation](https://mlsanity.com/paper/2511.19342/) (2025)

## Abstract (arXiv)

Many of the music generation systems based on neural networks are fully autonomous and do not offer control over the generation process. In this research, we present a controllable music generation system in terms of tonal tension. We incorporate two tonal tension measures based on the Spiral Array Tension theory into a variational autoencoder model. This allows us to control the direction of the tonal tension throughout the generated piece, as well as the overall level of tonal tension. Given a seed musical fragment, stemming from either the user input or from directly sampling from the latent space, the model can generate variations of this original seed fragment with altered tonal tension. This altered music still resembles the seed music rhythmically, but the pitch of the notes are changed to match the desired tonal tension as conditioned by the user.