# Generative Multi-modal Feedback for Singing Voice Synthesis Evaluation

arXiv: https://arxiv.org/abs/2512.02523 · HTML: https://mlsanity.com/paper/2512.02523/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Preference alignment for recommenders and RAG, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 20 of its 60 references that are in the map.
- First posted on arXiv 2 Dec 2025.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Preference alignment for recommenders and RAG](https://mlsanity.com/alignment/preference-alignment-for-recommenders-and-rag/)

## Builds on (most cited first, 20 of 20)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Qwen2.5-Omni Technical Report](https://mlsanity.com/paper/2503.20215/) (2025)
- [Qwen2-Audio Technical Report](https://mlsanity.com/paper/2407.10759/) (2024)
- [Kimi-Audio Technical Report](https://mlsanity.com/paper/2504.18425/) (2025)
- [DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism](https://mlsanity.com/paper/2105.02446/) (2021)
- [POP909: A Pop-song Dataset for Music Arrangement Generation](https://mlsanity.com/paper/2008.07142/) (2020)
- [MidiNet: A Convolutional Generative Adversarial Network for Symbolic-domain Music Generation](https://mlsanity.com/paper/1703.10847/) (2017)
- [Separate Anything You Describe](https://mlsanity.com/paper/2308.05037/) (2023)
- [VISinger: Variational Inference with Adversarial Learning for End-to-End Singing Voice Synthesis](https://mlsanity.com/paper/2110.08813/) (2021)
- [MusicRL: Aligning Music Generation to Human Preferences](https://mlsanity.com/paper/2402.04229/) (2024)
- [MorpheuS: generating structured music with constrained patterns and tension](https://mlsanity.com/paper/1812.04832/) (2018)
- [BATON: Aligning Text-to-Audio Model with Human Preference Feedback](https://mlsanity.com/paper/2402.00744/) (2024)
- [A Preliminary Exploration with GPT-4o Voice Mode](https://mlsanity.com/paper/2502.09940/) (2025)
- [MeloTrans: A Text to Symbolic Music Generation Model Following Human Composition Habit](https://mlsanity.com/paper/2410.13419/) (2024)
- [SiFiSinger: A High-Fidelity End-to-End Singing Voice Synthesizer based on Source-filter Model](https://mlsanity.com/paper/2410.12536/) (2024)
- [TG-Critic: A Timbre-Guided Model for Reference-Independent Singing Evaluation](https://mlsanity.com/paper/2305.09127/) (2023)
- [VISinger2+: End-to-End Singing Voice Synthesis Augmented by Self-Supervised Learning Representation](https://mlsanity.com/paper/2406.08761/) (2024)

## Abstract (arXiv)

Singing voice synthesis (SVS) has advanced significantly, enabling models to generate vocals with accurate pitch and consistent style. As these capabilities improve, the need for reliable evaluation and optimization becomes increasingly critical. However, current methods like reward systems often rely on single numerical scores, struggle to capture various dimensions such as phrasing or expressiveness, and require costly annotations, limiting interpretability and generalization. To address these issues, we propose a generative feedback (i.e., reward model) framework that provides multi-dimensional language and audio feedback for SVS assessment. Our approach leverages an audio-language model to generate text and audio critiques-covering aspects such as melody, content, and auditory quality. The model is fine-tuned on a hybrid dataset combining human music reactions and synthetic critiques from a MLLMs, enhancing diversity and linguistic richness. Quantitative experiments validate the effectiveness of the proposed dataset and training strategy, demonstrating that the framework produces musically accurate and interpretable evaluations suitable for guiding generative model improvement. The code is at [https://github.com/opendilab/VocalCritic](https://github.com/opendilab/VocalCritic)