# Amadeus: Autoregressive Model with Bidirectional Attribute Modelling for Symbolic Music

arXiv: https://arxiv.org/abs/2508.20665 · HTML: https://mlsanity.com/paper/2508.20665/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 39 references that are in the map.
- First posted on arXiv 28 Aug 2025.

## Builds on (most cited first, 24 of 24)

- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation](https://mlsanity.com/paper/2211.06687/) (2022)
- [Jukebox: A Generative Model for Music](https://mlsanity.com/paper/2005.00341/) (2020)
- [Scaling up Masked Diffusion Models on Text](https://mlsanity.com/paper/2410.18514/) (2024)
- [MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers](https://mlsanity.com/paper/2305.07185/) (2023)
- [Byte Latent Transformer: Patches Scale Better Than Tokens](https://mlsanity.com/paper/2412.09871/) (2024)
- [C-RNN-GAN: Continuous recurrent neural networks with adversarial training](https://mlsanity.com/paper/1611.09904/) (2016)
- [Compound Word Transformer: Learning to Compose Full-Song Music over Dynamic Directed Hypergraphs](https://mlsanity.com/paper/2101.02402/) (2021)
- [Mustango: Toward Controllable Text-to-Music Generation](https://mlsanity.com/paper/2311.08355/) (2023)
- [Diffuse and Disperse: Image Generation with Representation Regularization](https://mlsanity.com/paper/2506.09027/) (2025)
- [The Jazz Transformer on the Front Line: Exploring the Shortcomings of AI-composed Music through Quantitative Measures](https://mlsanity.com/paper/2008.01307/) (2020)
- [MuseCoco: Generating Symbolic Music from Text](https://mlsanity.com/paper/2306.00110/) (2023)
- [Symphony Generation with Permutation Invariant Language Model](https://mlsanity.com/paper/2205.05448/) (2022)
- [MuPT: A Generative Symbolic Music Pretrained Transformer](https://mlsanity.com/paper/2404.06393/) (2024)
- [NotaGen: Advancing Musicality in Symbolic Music Generation with Large Language Model Training Paradigms](https://mlsanity.com/paper/2502.18008/) (2025)
- [Text2midi: Generating Symbolic Music from Captions](https://mlsanity.com/paper/2412.16526/) (2024)
- [Discrete Diffusion Probabilistic Models for Symbolic Music Generation](https://mlsanity.com/paper/2305.09489/) (2023)
- [XMusic: Towards a Generalized and Controllable Symbolic Music Generation Framework](https://mlsanity.com/paper/2501.08809/) (2025)
- [MelodyT5: A Unified Score-to-Score Transformer for Symbolic Music Processing](https://mlsanity.com/paper/2407.02277/) (2024)
- [A database linking piano and orchestral MIDI scores with application to automatic projective orchestration](https://mlsanity.com/paper/1810.08611/) (2018)
- [Aria-MIDI: A Dataset of Piano MIDI Files for Symbolic Music Modeling](https://mlsanity.com/paper/2504.15071/) (2025)
- [The GigaMIDI Dataset with Features for Expressive Music Performance Detection](https://mlsanity.com/paper/2502.17726/) (2025)
- [Text2midi-InferAlign: Improving Symbolic Music Generation with Inference-Time Alignment](https://mlsanity.com/paper/2505.12669/) (2025)

## Abstract (arXiv)

Existing state-of-the-art symbolic music generation models predominantly adopt autoregressive or hierarchical autoregressive architectures, modelling symbolic music as a sequence of attribute tokens with unidirectional temporal dependencies, under the assumption of a fixed, strict dependency structure among these attributes. However, we observe that using different attributes as the initial token in these models leads to comparable performance. This suggests that the attributes of a musical note are, in essence, a concurrent and unordered set, rather than a temporally dependent sequence. Based on this insight, we introduce Amadeus, a novel symbolic music generation framework. Amadeus adopts a two-level architecture: an autoregressive model for note sequences and a bidirectional discrete diffusion model for attributes. To enhance performance, we propose Music Latent Space Discriminability Enhancement Strategy(MLSDES), incorporating contrastive learning constraints that amplify discriminability of intermediate music representations. The Conditional Information Enhancement Module (CIEM) simultaneously strengthens note latent vector representation via attention mechanisms, enabling more precise note decoding. We conduct extensive experiments on unconditional and text-conditioned generation tasks. Amadeus significantly outperforms SOTA models across multiple metrics while achieving at least 4× speed-up. Furthermore, we demonstrate training-free, fine-grained note attribute control feasibility using our model. To explore the upper performance bound of the Amadeus architecture, we compile the largest open-source symbolic music dataset to date, AMD (Amadeus MIDI Dataset), supporting both pre-training and fine-tuning.