mlsanityData to 14 Oct 2026

SDiaReward: Modeling and Benchmarking Spoken Dialogue Rewards with Modality and Colloquialness

arXiv 2603.14889 ↗First posted 16 Mar 2026eess.AS cs.CL cs.LGcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Preference alignment for recommenders and RAG

Abstract (arXiv)

The rapid evolution of end-to-end spoken dialogue systems demands transcending mere textual semantics to incorporate paralinguistic nuances and the spontaneous nature of human conversation. However, current methods struggle with two critical gaps: the modality gap, involving prosody and emotion, and the colloquialness gap, distinguishing written scripts from natural speech. To address these challenges, we introduce SDiaReward, an end-to-end multi-turn reward model trained on SDiaReward-Dataset, a novel collection of episode-level preference pairs explicitly targeting these gaps. It operates directly on full multi-turn speech episodes and is optimized with pairwise preference supervision, enabling joint assessment of modality and colloquialness in a single evaluator. We further establish ESDR-Bench, a stratified benchmark for robust episode-level evaluation. Experiments demonstrate that SDiaReward achieves state-of-the-art pairwise preference accuracy, significantly outperforming general-purpose audio LLMs. Further analysis suggests that SDiaReward captures relative conversational expressiveness beyond superficial synthesis cues, improving generalization across domains and recording conditions. Code, data, and demos are available at https://github.com/MM-Speech/SDiaReward/.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Preference alignment for recommenders and RAG, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 25 of its 54 references that are in the map. First posted on arXiv 16 Mar 2026.

Builds on

25 of 54 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025SpeechJudge: Towards Human-Level Judgment for Speech Naturalness11 Nov
  2. 2025WavReward: Spoken Dialogue Models With Generalist Reward Evaluators14 May

Other references · 23

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  5. 2017Deep reinforcement learning from human preferencescited by 3,912
  6. 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 643
  7. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
  8. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 564
  9. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 529
  10. 2024Mini-Omni: Language Models Can Hear, Talk While Thinking in Streamingcited by 210
  11. 2024VoiceBench: Benchmarking LLM-Based Voice Assistantscited by 200
  12. 2024RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Stylecited by 176
11 more
  1. 2022DailyTalk: Spoken Dialogue Dataset for Conversational Text-to-Speechcited by 69
  2. 2025FireRedTTS-2: Towards Long Conversational Speech Generation for Podcast and Chatbotcited by 35
  3. 2025VITA-Audio: Fast Interleaved Cross-Modal Token Generation for Efficient Large Speech-Language Modelcited by 31
  4. 2024GTSinger: A Global Multi-Technique Singing Corpus with Realistic Music Scores for All Singing Taskscited by 30
  5. 2025URO-Bench: Towards Comprehensive Evaluation for End-to-End Spoken Dialogue Modelscited by 30
  6. 2025ParaS2S: Benchmarking and Aligning Spoken Language Models for Paralinguistic-aware Speech-to-Speech Interactioncited by 24
  7. 2025WavRAG: Audio-Integrated Retrieval Augmented Generation for Spoken Dialogue Modelscited by 18
  8. 2025OmniChat: Enhancing Spoken Dialogue Systems with Scalable Synthetic Data for Diverse Scenarioscited by 9
  9. 2025SoulX-Podcast: Towards Realistic Long-form Podcasts with Dialectal and Paralinguistic Diversitycited by 6
  10. 2026WavBench: Benchmarking Reasoning, Colloquialism, and Paralinguistics for End-to-End Spoken Dialogue Modelscited by 6
  11. 2026Synthetic Singers: A Review of Deep-Learning-based Singing Voice Synthesis Approachescited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Optimizing Conversational Quality in Spoken Dialogue Systems with Reinforcement Learning from AI Feedback
  2. 2025Aligning Spoken Dialogue Models from User Interactions
  3. 2026WavAlign: Enhancing Intelligence and Expressiveness in Spoken Dialogue Models via Adaptive Hybrid Post-Training
  4. 2025Aligning Dialogue Agents with Global Feedback via Large Language Model Multimodal Reward Decomposition
  5. 2025Learning LLM Preference over Intra-Dialogue Pairs: A Framework for Utterance-level Understandings
  6. 2024Improving Dialogue Agents by Decomposing One Global Explicit Annotation with Local Implicit Multimodal Feedback
  7. 2026Dual-Axis Generative Reward Model Toward Semantic and Turn-taking Robustness in Interactive Spoken Dialogue Models
  8. 2025Audio MultiChallenge: A Multi-Turn Evaluation of Spoken Dialogue Systems on Natural Human Interaction