mlsanityData to 7 Oct 2026

Fine-grained Video Dubbing Duration Alignment with Segment Supervised Preference Optimization

arXiv 2508.08550 ↗First posted 12 Aug 2025cs.SD cs.CLcited by 3

On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment

Abstract (arXiv)

Video dubbing aims to translate original speech in visual media programs from the source language to the target language, relying on neural machine translation and text-to-speech technologies. Due to varying information densities across languages, target speech often mismatches the source speech duration, causing audio-video synchronization issues that significantly impact viewer experience. In this study, we approach duration alignment in LLM-based video dubbing machine translation as a preference optimization problem. We propose the Segment Supervised Preference Optimization (SSPO) method, which employs a segment-wise sampling strategy and fine-grained loss to mitigate duration mismatches between source and target lines. Experimental results demonstrate that SSPO achieves superior performance in duration alignment tasks.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 39 of its 64 references that are in the map. First posted on arXiv 12 Aug 2025.

Builds on

39 of 64 references.

References · 39

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  7. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  8. 2023Qwen Technical Reportcited by 3,273
  9. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  10. 2020Learning to summarize from human feedbackcited by 2,562
  11. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  12. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
27 more
  1. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  2. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  3. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  4. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 906
  5. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  6. 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 260
  7. 2023Secrets of RLHF in Large Language Models Part I: PPOcited by 226
  8. 2020Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPOcited by 184
  9. 2021A Survey on Neural Speech Synthesiscited by 182
  10. 2017Interactive Learning from Policy-Dependent Human Feedbackcited by 155
  11. 2018Correcting Length Bias in Neural Machine Translationcited by 119
  12. 2016Controlling Output Length in Neural Encoder-Decoderscited by 75
  13. 2019Positional Encoding to Control Output Sequence Lengthcited by 56
  14. 2020Handwritten Optical Character Recognition (OCR): A Comprehensive Systematic Literature Review (SLR)cited by 56
  15. 2024Controllable Text Generation for Large Language Models: A Surveycited by 55
  16. 2016Semi-Supervised Learning for Neural Machine Translationcited by 36
  17. 2019Controlling the Output Length of Neural Machine Translationcited by 36
  18. 2020Semi-supervised ASR by End-to-end Self-trainingcited by 29
  19. 2025SDPO: Segment-Level Direct Preference Optimization for Social Agentscited by 29
  20. 2020From Speech-to-Speech Translation to Automatic Dubbingcited by 25
  21. 2024How Good Are LLMs for Literary Translation, Really? Literary Translation Evaluation with Humans and LLMscited by 23
  22. 2022VideoDubber: Machine Translation with Speech-Aware Length Control for Video Dubbingcited by 20
  23. 2024When Text Embedding Meets Large Language Model: A Comprehensive Surveycited by 19
  24. 2021LenAtten: An Effective Length Controlling Unit For Text Summarizationcited by 14
  25. 2025Propagation Tree Is Not Deep: Adaptive Graph Contrastive Learning Approach for Rumor Detectioncited by 7
  26. 2025Enhancing Rumor Detection Methods with Propagation Structure Infused Language Modelcited by 2
  27. 2025Towards Real-World Rumor Detection: Anomaly Detection Framework with Graph Supervised Contrastive Learningcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026JUST-DUB-IT: Video Dubbing via Joint Audio-Visual Diffusioncited by 6
  2. 2026Hermes the Polyglot: A Unified Framework to Enhance Expressiveness for Multimodal Interlingual Subtitlingcited by 2
  3. 2026CineSRD: Leveraging Visual, Acoustic, and Linguistic Cues for Open-World Visual Media Speaker Diarizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025TEMPLE: Incentivizing Temporal Understanding of Video Large Language Models via Progressive Pre-SFT Alignment
  2. 2025Self-alignment of Large Video Language Models with Refined Regularized Preference Optimization
  3. 2026SyncDPO: Enhancing Temporal Synchronization in Video-Audio Joint Generation via Preference Learning
  4. 2026FunCineForge: A Unified Dataset Toolkit and Model for Zero-Shot Movie Dubbing in Diverse Cinematic Scenes
  5. 2025DPO-Tuned Large Language Models for Segmentation in Simultaneous Speech Translation
  6. 2026CoSyncDiT: Cognitive Synchronous Diffusion Transformer for Movie Dubbing
  7. 2025Temporal Preference Optimization for Long-Form Video Understanding
  8. 2026Align Then Reason: A Multimodal Lip-Sync Judge for Dubbing