# Music Arena: Live Evaluation for Text-to-Music

arXiv: https://arxiv.org/abs/2507.20900 · HTML: https://mlsanity.com/paper/2507.20900/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 29 references that are in the map.
- First posted on arXiv 28 Jul 2025.

## Builds on (most cited first, 18 of 18)

- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [Jukebox: A Generative Model for Music](https://mlsanity.com/paper/2005.00341/) (2020)
- [Simple and Controllable Music Generation](https://mlsanity.com/paper/2306.05284/) (2023)
- [Stable Audio Open](https://mlsanity.com/paper/2407.14358/) (2024)
- [Gradio: Hassle-Free Sharing and Testing of ML Models in the Wild](https://mlsanity.com/paper/1906.02569/) (2019)
- [Adapting Frechet Audio Distance for Generative Music Evaluation](https://mlsanity.com/paper/2311.01616/) (2023)
- [Long-form music generation with latent diffusion](https://mlsanity.com/paper/2404.10301/) (2024)
- [GenAI Arena: An Open Evaluation Platform for Generative Models](https://mlsanity.com/paper/2406.04485/) (2024)
- [ACE-Step: A Step Towards Music Generation Foundation Model](https://mlsanity.com/paper/2506.00045/) (2025)
- [MusicRL: Aligning Music Generation to Human Preferences](https://mlsanity.com/paper/2402.04229/) (2024)
- [SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generation](https://mlsanity.com/paper/2502.13128/) (2025)
- [Copilot Arena: A Platform for Code LLM Evaluation in the Wild](https://mlsanity.com/paper/2502.09328/) (2025)
- [Aligning Text-to-Music Evaluation with Human Preferences](https://mlsanity.com/paper/2503.16669/) (2025)
- [The Ethical Implications of Generative Audio Models: A Systematic Literature Review](https://mlsanity.com/paper/2307.05527/) (2023)
- [MusicEval: A Generative Music Dataset with Expert Ratings for Automatic Text-to-Music Evaluation](https://mlsanity.com/paper/2501.10811/) (2025)
- [Benchmarking Music Generation Models and Metrics via Human Preference Studies](https://mlsanity.com/paper/2506.19085/) (2025)
- [Fast Text-to-Audio Generation with Adversarial Post-Training](https://mlsanity.com/paper/2505.08175/) (2025)
- [3D Arena: An Open Platform for Generative 3D Evaluation](https://mlsanity.com/paper/2506.18787/) (2025)

## Built on by (most cited first, 8 of 8)

- [CMI-RewardBench: Evaluating Music Reward Models with Compositional Multimodal Instruction](https://mlsanity.com/paper/2603.00610/) (2026)
- [TuneJury: An Open Metric for Improving Music Generation Preference Alignment](https://mlsanity.com/paper/2606.17006/) (2026)
- [When Noise Lowers The Loss: Rethinking Likelihood-Based Evaluation in Music Large Language Models](https://mlsanity.com/paper/2602.02738/) (2026)
- [APEX: Large-scale Multi-task Aesthetic-Informed Popularity Prediction for AI-Generated Music](https://mlsanity.com/paper/2605.03395/) (2026)
- [Improving Text-to-Music Generation with Human Preference Rewards](https://mlsanity.com/paper/2606.21670/) (2026)
- [MuseCritic: Learning Multi-Aspect Song Rewards through Natural-Language Aesthetic Critiques](https://mlsanity.com/paper/2608.11755/) (2026)
- [TTM-Bench: A Framework for Text-to-Music System Performance Benchmarking](https://mlsanity.com/paper/2609.18585/) (2026)
- [Don't CLAP: Are Music-Text Models Bag-of-Words?](https://mlsanity.com/paper/2609.30540/) (2026)

## Abstract (arXiv)

We present Music Arena, an open platform for scalable human preference evaluation of text-to-music (TTM) models. Soliciting human preferences via listening studies is the gold standard for evaluation in TTM, but these studies are expensive to conduct and difficult to compare, as study protocols may differ across systems. Moreover, human preferences might help researchers align their TTM systems or improve automatic evaluation metrics, but an open and renewable source of preferences does not currently exist. We aim to fill these gaps by offering *live* evaluation for TTM. In Music Arena, real-world users input text prompts of their choosing and compare outputs from two TTM systems, and their preferences are used to compile a leaderboard. While Music Arena follows recent evaluation trends in other AI domains, we also design it with key features tailored to music: an LLM-based routing system to navigate the heterogeneous type signatures of TTM systems, and the collection of *detailed* preferences including listening data and natural language feedback. We also propose a rolling data release policy with user privacy guarantees, providing a renewable source of preference data and increasing platform transparency. Through its standardized evaluation protocol, transparent data access policies, and music-specific features, Music Arena not only addresses key challenges in the TTM ecosystem but also demonstrates how live evaluation can be thoughtfully adapted to unique characteristics of specific AI domains. Music Arena is available at: https://music-arena.org . Preference data is available at: https://huggingface.co/music-arena .