# SMART: Tuning a symbolic music generation system with an audio domain aesthetic reward

arXiv: https://arxiv.org/abs/2504.16839 · HTML: https://mlsanity.com/paper/2504.16839/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Apr 2025.

## Built on by (most cited first, 1 of 1)

- [Which Constraints Are Missing? Ask the Verifier: Graded Rewards for Constraint-Following Music Generation](https://mlsanity.com/paper/2609.23665/) (2026)

## Abstract (arXiv)

Recent work has proposed training machine learning models to predict aesthetic ratings for music audio. Our work explores whether such models can be used to finetune a symbolic music generation system with reinforcement learning, and what effect this has on the system outputs. To test this, we use group relative policy optimization to finetune a piano MIDI model with Meta Audiobox Aesthetics ratings of audio-rendered outputs as the reward. We find that this optimization has effects on multiple low-level features of the generated outputs, and improves the average subjective ratings in a preliminary listening study with 14 participants. We also find that over-optimization dramatically reduces diversity of model outputs.