mlsanityData to 7 Oct 2026

SpeechCritic: Learning a Diagnostic Speech Judge from Limited Human Preferences

arXiv 2609.34582 ↗First posted 28 Sep 2026cs.AI cs.SDcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Preference alignment for recommenders and RAG also fits: Medical LLM alignment

Abstract (arXiv)

Human speech conveys rich perceptual information, such as emotion and speaker identity, yet most automatic speech quality judges reduce it to a single naturalness score. We study diagnostic speech judges: given two candidates, a diagnostic judge decides which is better, along which perceptual dimensions (e.g., timbre, emotion, timing) they differ, and which audible cues support its decision. Learning such judges is challenging: expert annotation is costly, and simply prompting a frontier audio-language model to produce labels is unreliable: our probing reveals substantial errors and unstable instruction following. We introduce SpeechCritic, which learns a diagnostic judge in a reference-conditioned cross-lingual setting from only about 300 human-labeled comparisons. Rather than replacing the frontier model, SpeechCritic calibrates it with these labels: for each dimension, it selects the acoustic measurements that agree with human judgments, maps them to A/Tie/B probabilities, and passes these to the model as non-binding hints alongside the audio. Compared with the same model labeling without hints, this raises dimension-level agreement with humans by 6.3 points and cuts the mismatch with human Tie rates by 10.4 points. We then train a 7B judge on this supervision and find that different training signals shape different judge behaviors: SFT establishes the task, OPD transfers the teacher's dimension-level strengths and weaknesses, and RL helps most on clear-cut comparisons where human raters agree. Notably, human listeners also find that RL makes rationales cite more specific, localized acoustic cues, although it never directly rewards rationale text. Finally, we show that the pipeline is language-pair agnostic by instantiating it on both English-Japanese and English-Spanish. Together, these results demonstrate a path from limited human preferences to a diagnostic speech judge.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Preference alignment for recommenders and RAG, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 28 of its 43 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

28 of 43 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026GSRM: Generative Speech Reward Model for Speech RLHF14 Feb
  2. 2025SpeechJudge: Towards Human-Level Judgment for Speech Naturalness11 Nov
  3. 2025WavReward: Spoken Dialogue Models With Generalist Reward Evaluators14 May

Other references · 25

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
  4. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 615
  5. 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 558
  6. 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelscited by 386
  7. 2023JudgeLM: Fine-tuned Large Language Models are Scalable Judgescited by 351
  8. 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 345
  9. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 282
  10. 2020DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressorscited by 235
  11. 2025Kimi-Audio Technical Reportcited by 210
  12. 2023emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representationcited by 196
13 more
  1. 2021NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasetscited by 189
  2. 2023Generative Judge for Evaluating Alignmentcited by 163
  3. 2022Dawn of the transformer era in speech emotion recognition: closing the valence gapcited by 155
  4. 2020VoxLingua107: a Dataset for Spoken Language Recognitioncited by 118
  5. 2026Qwen3-TTS Technical Reportcited by 106
  6. 2024Critique-out-Loud Reward Modelscited by 105
  7. 2022Wespeaker: A Research and Production oriented Speaker Embedding Learning Toolkitcited by 92
  8. 2025AudioJudge: Understanding What Works in Large Audio Model Based Speech Evaluationcited by 20
  9. 2025SpeechLLM-as-Judges: Towards General and Interpretable Speech Quality Evaluationcited by 14
  10. 2025QualiSpeech: A Speech Quality Assessment Dataset with Natural Language Reasoning and Descriptionscited by 13
  11. 2026Hearing Between the Lines: Unlocking the Reasoning Power of LLMs for Speech Evaluationcited by 5
  12. 2026UniSRM: A Unified Speech Reward Model for Reasoning-Based Fine-grained Assessmentcited by 2
  13. 2026JASTIN: Aligning LLMs for Zero-Shot Audio and Speech Evaluation via Natural Language Instructionscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025SpeechR: A Benchmark for Speech Reasoning in Large Audio-Language Models
  2. 2025Decoding the Ear: A Framework for Objectifying Expressiveness from Human Preference Through Efficient Alignment
  3. 2026DEAF: A Benchmark for Diagnostic Evaluation of Acoustic Faithfulness in Audio Language Models
  4. 2026HEAR Who Said What: Unlocking Speaker-Attributed Reasoning via Counterfactual Voice Grounding
  5. 2026AURAL: Adaptive Latent Reasoning with Joint Chunk for Speech Language Models
  6. 2025Teaching Audio Models to Reason: A Unified Framework for Source- and Layer-wise Distillation
  7. 2026When Does Predictor-Based RL Align with Human Perception? A Study of Subjective Rewards in Codec-Based Speech Language Models
  8. 2026AnyAudio-Judge: A Dynamic Rubric-Based Benchmark and Evaluator for Audio Instruction Following