mlsanityData to 7 Oct 2026

LLM-Based Multi-Reference Evaluation for Efficient and Robust Assessment of Phrase Break Annotations

arXiv 2606.21098 ↗First posted 19 Jun 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Reliable evaluation of phrase break annotations is crucial, as subtle variations in prosodic boundaries directly affect the clarity and naturalness of speech. However, existing approaches exhibit major limitations: single-reference evaluation assumes a unique gold phrasing for an utterance despite multiple valid phrasings, while human judgment, though flexible, is labor-intensive and unscalable. To address these, we propose LLM-based Multi-Reference Evaluation (LMRE) for phrase break annotations that models the one-to-many nature of prosodic phrasing and generates multiple valid phrasings from minimal demonstrations. On a Korean testbed of 1,356 annotations covering five strategies, LMRE shows stronger alignment with human judgment than single-reference evaluation in both acceptance behavior and score correlation. Our findings demonstrate that LMRE effectively achieves both scalability and multi-reference support, highlighting the potential of LLMs for evaluation in the speech domain.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 17 of its 31 references that are in the map. First posted on arXiv 19 Jun 2026.

Builds on

17 of 31 references.

References · 17

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  3. 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 558
  4. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 416
  5. 2023Batch Prompting: Efficient Inference with Large Language Model APIscited by 84
  6. 2023On Learning to Summarize with Large Language Models as Referencescited by 80
  7. 2024RevisEval: Improving LLM-as-a-Judge via Response-Adapted Referencescited by 19
  8. 2021Phrase break prediction with bidirectional encoder representations in Japanese text-to-speech synthesiscited by 12
  9. 2023Not All Metrics Are Guilty: Improving NLG Evaluation by Diversifying Referencescited by 11
  10. 2020Modeling Prosodic Phrasing with Multi-Task Learning in Tacotron-based TTScited by 6
  11. 2023Duration-aware pause insertion using pre-trained language model for multi-speaker text-to-speechcited by 5
  12. 2023Assessing Phrase Break of ESL Speech with Pre-trained Language Models and Large Language Modelscited by 5
5 more
  1. 2023PauseSpeech: Natural Speech Synthesis via Pre-trained Language Model and Pause-based Prosody Modelingcited by 5
  2. 2023Cross-Lingual Transfer Learning for Phrase Break Prediction with Multilingual Language Modelcited by 2
  3. 2024ProsodyFM: Unsupervised Phrasing and Intonation Control for Intelligible Speech Synthesiscited by 2
  4. 2025Synthetic Data Generation for Phrase Break Prediction with Large Language Modelcited by 2
  5. 2025A Linguistically Motivated Analysis of Intonational Phrasing in Text-to-Speech Systems: Revealing Gaps in Syntactic Sensitivitycited by 1

Built on by

Not cited yet.