mlsanityData to 7 Oct 2026

Speech-Worthy Alignment for Japanese SpeechLLMs via Direct Preference Optimization

arXiv 2603.12565 ↗First posted 13 Mar 2026cs.SD cs.CLcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Preference alignment for recommenders and RAG

Abstract (arXiv)

SpeechLLMs typically combine ASR-trained encoders with text-based LLM backbones, leading them to inherit written-style output patterns unsuitable for text-to-speech synthesis. This mismatch is particularly pronounced in Japanese, where spoken and written registers differ substantially in politeness markers, sentence-final particles, and syntactic complexity. We propose a preference-based alignment approach to adapt Japanese SpeechLLMs for speech-worthy outputs: text that is concise, conversational, and readily synthesized as natural speech. To rigorously evaluate this task, we introduce SpokenElyza, a benchmark for Japanese speech-worthiness derived from ELYZA-tasks-100 with auditory verification by native experts. Experiments show that our approach achieves substantial improvement on SpokenElyza while largely preserving performance on the original written-style evaluation. We will release SpokenElyza to support future research on Japanese spoken dialog systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Preference alignment for recommenders and RAG, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 18 of its 21 references that are in the map. First posted on arXiv 13 Mar 2026.

Builds on

18 of 21 references.

References · 18

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  4. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  5. 2024Qwen2-Audio Technical Reportcited by 543
  6. 2023SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilitiescited by 518
  7. 2023SALMONN: Towards Generic Hearing Abilities for Large Language Modelscited by 492
  8. 2023Listen, Think, and Understandcited by 206
  9. 2024VoiceBench: Benchmarking LLM-Based Voice Assistantscited by 192
  10. 2024LLaMA-Omni: Seamless Speech Interaction with Large Language Modelscited by 169
  11. 2024GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilitiescited by 151
  12. 2024SLAM-Omni: Timbre-Controllable Voice Interaction System with Single-Stage Trainingcited by 58
6 more
  1. 2025Music Flamingo: Scaling Music Understanding in Audio Language Modelscited by 19
  2. 2024OpenMU: Your Swiss Army Knife for Music Understandingcited by 9
  3. 2025DeepDialogue: A Multi-Turn Emotionally-Rich Spoken Dialogue Datasetcited by 8
  4. 2024Speechworthy Instruction-tuned Language Modelscited by 7
  5. 2025DeepResonance: Enhancing Multimodal Music Understanding via Music-centric Multi-way Instruction Tuningcited by 2
  6. 2025Think, Verbalize, then Speak: Bridging Complex Thoughts and Comprehensible Speechcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Data-efficient Targeted Token-level Preference Optimization for LLM-based Text-to-Speech
  2. 2026Ready to Speak: Aligning LLMs for TTS-Friendly Text Generation
  3. 2024Preference Alignment Improves Language Model-Based TTS
  4. 2025Aligning Spoken Dialogue Models from User Interactions
  5. 2024SpeechAlign: Aligning Speech Generation to Human Preferences
  6. 2024Comparing Bad Apples to Good Oranges: Aligning Large Language Models via Joint Preference Optimization
  7. 2026Bridging the Stability-Expressivity Gap: Synthetic Data Scaling and Preference Alignment for Low-Resource Spoken Language Models
  8. 2025No Verifiable Reward for Prosody: Toward Preference-Guided Prosody Learning in TTS