ViTex: Visual Texture Control for Multi-Track Symbolic Music Generation via Discrete Diffusion Models
On the map: Diffusion models → Audio, speech and music
Abstract (arXiv)
In automatic music generation, a central challenge is to design controls that enable meaningful human-machine interaction. Existing systems often rely on extrinsic inputs such as text prompts or metadata, which do not allow humans to directly shape the composition. While prior work has explored intrinsic controls such as chords or hierarchical structure, these approaches mainly address piano or vocal-accompaniment settings, leaving multitrack symbolic music largely underexplored. We identify instrumentation, the choice of instruments and their roles, as a natural dimension of control in multi-track composition, and propose ViTex, a visual representation of instrumental texture. In ViTex, color encodes instrument choice, spatial position represents pitch and time, and stroke properties capture local textures. Building on this representation, we develop a discrete diffusion model conditioned on ViTex and chord progressions to generate 8-measure multi-track symbolic music, enabling explicit texture-level control while maintaining strong unconditional generation quality. The demo page and code are avaliable at https://vitex2025.github.io/.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 19 of its 27 references that are in the map. First posted on arXiv 2 Mar 2026.
Builds on
19 of 27 references.
Closest ancestors · 3
Same direction, newest first
- 2024Whole-Song Hierarchical Generation of Symbolic Music Using Cascaded Diffusion Models16 May
- 2023Polyffusion: A Diffusion Model for Polyphonic Score Generation with Internal and External Controls19 Jul
- 2021Symbolic Music Generation with Diffusion Models30 Mar
Other references · 16
- 2017Attention Is All You Needcited by 59,932
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2022Classifier-Free Diffusion Guidancecited by 5,717
- 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
- 2020PopMAG: Pop Music Accompaniment Generationcited by 68
- 2020The Jazz Transformer on the Front Line: Exploring the Shortcomings of AI-composed Music through Quantitative Measurescited by 53
- 2023MuseCoco: Generating Symbolic Music from Textcited by 50
- 2020Learning Interpretable Representation for Controllable Polyphonic Music Generationcited by 40
- 2020MusPy: A Toolkit for Symbolic Music Generationcited by 37
- 2023Anticipatory Music Transformercited by 35
- 2019Deep Music Analogy Via Latent Representation Disentanglementcited by 31
4 more
- 2020Music SketchNet: Controllable Music Generation via Factorized Representations of Pitch and Rhythmcited by 28
- 2022Multitrack Music Transformercited by 28
- 2021AccoMontage: Accompaniment Arrangement via Phrase Selection and Style Transfercited by 22
- 2023Q&A: Query-Based Representation Learning for Multi-Track Symbolic Music re-Arrangementcited by 7
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024Pictures Of MIDI: Controlled Music Generation via Graphical Prompts for Image-Based Diffusion Inpainting
- 2023DiffuseRoll: Multi-track multi-category music generation based on diffusion model
- 2024Efficient Fine-Grained Guidance for Diffusion Model Based Symbolic Music Generation
- 2024Editing Music with Melody and Text: Using ControlNet for Diffusion Transformer
- 2024Multi-Track MusicLDM: Towards Versatile Music Generation with Latent Diffusion Model
- 2024Symbolic Music Generation with Non-Differentiable Rule Guided Diffusion
- 2025Diff-TONE: Timestep Optimization for iNstrument Editing in Text-to-Music Diffusion Models
- 2026Art2Mus: Artwork-to-Music Generation via Visual Conditioning and Large-Scale Cross-Modal Alignment