# Beat-Aligned Spectrogram-to-Sequence Generation of Rhythm-Game Charts

arXiv: https://arxiv.org/abs/2311.13687 · HTML: https://mlsanity.com/paper/2311.13687/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 9 references that are in the map.
- First posted on arXiv 22 Nov 2023.

## Builds on (most cited first, 7 of 7)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Music Transformer](https://mlsanity.com/paper/1809.04281/) (2018)
- [MT3: Multi-Task Multitrack Music Transcription](https://mlsanity.com/paper/2111.03017/) (2021)
- [Dance Dance Convolution](https://mlsanity.com/paper/1703.06891/) (2017)
- [TaikoNation: Patterning-focused Chart Generation for Rhythm Action Games](https://mlsanity.com/paper/2107.12506/) (2021)
- [Gen'eLive! Generating Rhythm Actions in Love Live!](https://mlsanity.com/paper/2202.12823/) (2022)

## Built on by (most cited first, 1 of 1)

- [ITGPT: A Transformer Based Architecture for the Generation of Dance Dance Revolution and In the Groove Charts](https://mlsanity.com/paper/2607.14148/) (2026)

## Abstract (arXiv)

In the heart of "rhythm games" - games where players must perform actions in sync with a piece of music - are "charts", the directives to be given to players. We newly formulate chart generation as a sequence generation task and train a Transformer using a large dataset. We also introduce tempo-informed preprocessing and training procedures, some of which are suggested to be integral for a successful training. Our model is found to outperform the baselines on a large dataset, and is also found to benefit from pretraining and finetuning.