# ViTex: Visual Texture Control for Multi-Track Symbolic Music Generation via Discrete Diffusion Models

arXiv: https://arxiv.org/abs/2603.01984 · HTML: https://mlsanity.com/paper/2603.01984/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 19 of its 27 references that are in the map.
- First posted on arXiv 2 Mar 2026.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [Symbolic Music Generation with Diffusion Models](https://mlsanity.com/paper/2103.16091/) (2021)
- [PopMAG: Pop Music Accompaniment Generation](https://mlsanity.com/paper/2008.07703/) (2020)
- [The Jazz Transformer on the Front Line: Exploring the Shortcomings of AI-composed Music through Quantitative Measures](https://mlsanity.com/paper/2008.01307/) (2020)
- [MuseCoco: Generating Symbolic Music from Text](https://mlsanity.com/paper/2306.00110/) (2023)
- [Learning Interpretable Representation for Controllable Polyphonic Music Generation](https://mlsanity.com/paper/2008.07122/) (2020)
- [MusPy: A Toolkit for Symbolic Music Generation](https://mlsanity.com/paper/2008.01951/) (2020)
- [Anticipatory Music Transformer](https://mlsanity.com/paper/2306.08620/) (2023)
- [Polyffusion: A Diffusion Model for Polyphonic Score Generation with Internal and External Controls](https://mlsanity.com/paper/2307.10304/) (2023)
- [Deep Music Analogy Via Latent Representation Disentanglement](https://mlsanity.com/paper/1906.03626/) (2019)
- [Music SketchNet: Controllable Music Generation via Factorized Representations of Pitch and Rhythm](https://mlsanity.com/paper/2008.01291/) (2020)
- [Multitrack Music Transformer](https://mlsanity.com/paper/2207.06983/) (2022)
- [Whole-Song Hierarchical Generation of Symbolic Music Using Cascaded Diffusion Models](https://mlsanity.com/paper/2405.09901/) (2024)
- [AccoMontage: Accompaniment Arrangement via Phrase Selection and Style Transfer](https://mlsanity.com/paper/2108.11213/) (2021)
- [Q&A: Query-Based Representation Learning for Multi-Track Symbolic Music re-Arrangement](https://mlsanity.com/paper/2306.01635/) (2023)

## Built on by (most cited first, 1 of 1)

- [DDSynth-RL: Audio Synthesizer Inversion via Discrete Diffusion with Reinforcement Learning](https://mlsanity.com/paper/2608.03032/) (2026)

## Abstract (arXiv)

In automatic music generation, a central challenge is to design controls that enable meaningful human-machine interaction. Existing systems often rely on extrinsic inputs such as text prompts or metadata, which do not allow humans to directly shape the composition. While prior work has explored intrinsic controls such as chords or hierarchical structure, these approaches mainly address piano or vocal-accompaniment settings, leaving multitrack symbolic music largely underexplored. We identify instrumentation, the choice of instruments and their roles, as a natural dimension of control in multi-track composition, and propose ViTex, a visual representation of instrumental texture. In ViTex, color encodes instrument choice, spatial position represents pitch and time, and stroke properties capture local textures. Building on this representation, we develop a discrete diffusion model conditioned on ViTex and chord progressions to generate 8-measure multi-track symbolic music, enabling explicit texture-level control while maintaining strong unconditional generation quality. The demo page and code are avaliable at https://vitex2025.github.io/.