# RELATE: Subjective evaluation dataset for automatic evaluation of relevance between text and audio

arXiv: https://arxiv.org/abs/2506.23582 · HTML: https://mlsanity.com/paper/2506.23582/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jun 2025.

## Built on by (most cited first, 3 of 3)

- [AQAScore: Evaluating Semantic Alignment in Text-to-Audio Generation via Audio Question Answering](https://mlsanity.com/paper/2601.14728/) (2026)
- [MoEScore: Mixture-of-Experts-Based Text-Audio Relevance Score Prediction for Text-to-Audio System Evaluation](https://mlsanity.com/paper/2601.06829/) (2026)
- [CAF-Score: Calibrating CLAP with LALMs for Reference-free Audio Captioning Evaluation](https://mlsanity.com/paper/2603.19615/) (2026)

## Abstract (arXiv)

In text-to-audio (TTA) research, the relevance between input text and output audio is an important evaluation aspect. Traditionally, it has been evaluated from both subjective and objective perspectives. However, subjective evaluation is costly in terms of money and time, and objective evaluation is unclear regarding the correlation to subjective evaluation scores. In this study, we construct RELATE, an open-sourced dataset that subjectively evaluates the relevance. Also, we benchmark a model for automatically predicting the subjective evaluation score from synthesized audio. Our model outperforms a conventional CLAPScore model, and that trend extends to many sound categories.