# Temporal Preference Optimization for Long-Form Video Understanding

arXiv: https://arxiv.org/abs/2501.13919 · HTML: https://mlsanity.com/paper/2501.13919/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 58 of its 69 references that are in the map.
- First posted on arXiv 23 Jan 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Video understanding with MLLMs](https://mlsanity.com/mllm/video-understanding-with-mllms/)
- LLM alignment / RLHF / preference optimisation / [Diffusion and multimodal model alignment](https://mlsanity.com/alignment/diffusion-and-multimodal-model-alignment/)

## Builds on (most cited first, 30 of 58)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://mlsanity.com/paper/2404.14219/) (2024)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [MiniCPM-V: A GPT-4V Level MLLM on Your Phone](https://mlsanity.com/paper/2408.01800/) (2024)
- [STaR: Bootstrapping Reasoning With Reasoning](https://mlsanity.com/paper/2203.14465/) (2022)
- [VideoChat: Chat-Centric Video Understanding](https://mlsanity.com/paper/2305.06355/) (2023)
- [Aligning Large Multimodal Models with Factually Augmented RLHF](https://mlsanity.com/paper/2309.14525/) (2023)
- [VILA: On Pre-training for Visual Language Models](https://mlsanity.com/paper/2312.07533/) (2023)
- [LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding](https://mlsanity.com/paper/2407.15754/) (2024)
- [Large Language Models Can Self-Improve](https://mlsanity.com/paper/2210.11610/) (2022)
- [TALL: Temporal Activity Localization via Language Query](https://mlsanity.com/paper/1705.02101/) (2017)
- [Long Context Transfer from Language to Vision](https://mlsanity.com/paper/2406.16852/) (2024)
- [TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding](https://mlsanity.com/paper/2312.02051/) (2023)
- [ShareGPT4Video: Improving Video Understanding and Generation with Better Captions](https://mlsanity.com/paper/2406.04325/) (2024)
- [LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding](https://mlsanity.com/paper/2410.17434/) (2024)
- [LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Models](https://mlsanity.com/paper/2407.12772/) (2024)
- [VideoAgent: Long-form Video Understanding with Large Language Model as Agent](https://mlsanity.com/paper/2403.10517/) (2024)

## Built on by (most cited first, 29 of 29)

- [SmolVLM: Redefining small and efficient multimodal models](https://mlsanity.com/paper/2504.05299/) (2025)
- [Unified Reward Model for Multimodal Understanding and Generation](https://mlsanity.com/paper/2503.05236/) (2025)
- [TSPO: Temporal Sampling Policy Optimization for Long-form Video Language Understanding](https://mlsanity.com/paper/2508.04369/) (2025)
- [MR. Video: "MapReduce" is the Principle for Long Video Understanding](https://mlsanity.com/paper/2504.16082/) (2025)
- [FlexSelect: Flexible Token Selection for Efficient Long Video Understanding](https://mlsanity.com/paper/2506.00993/) (2025)
- [Hallo4: High-Fidelity Dynamic Portrait Animation via Direct Preference Optimization](https://mlsanity.com/paper/2505.23525/) (2025)
- [TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Action](https://mlsanity.com/paper/2505.01583/) (2025)
- [Agentic Keyframe Search for Video Question Answering](https://mlsanity.com/paper/2503.16032/) (2025)
- [AVATAR: Reinforcement Learning to See, Hear, and Reason Over Video](https://mlsanity.com/paper/2508.03100/) (2025)
- [SEASON: Mitigating Temporal Hallucination in Video Large Language Models via Self-Diagnostic Contrastive Decoding](https://mlsanity.com/paper/2512.04643/) (2025)
- [Tool-Augmented Spatiotemporal Reasoning for Streamlining Video Question Answering Task](https://mlsanity.com/paper/2512.10359/) (2025)
- [VideoPASTA: 7K Preference Pairs That Matter for Video-LLM Alignment](https://mlsanity.com/paper/2504.14096/) (2025)
- [Zoom-Zero: Reinforced Coarse-to-Fine Video Understanding via Temporal Zoom-in](https://mlsanity.com/paper/2512.14273/) (2025)
- [Self-alignment of Large Video Language Models with Refined Regularized Preference Optimization](https://mlsanity.com/paper/2504.12083/) (2025)
- [VideoVista-CulturalLingo: 360^circ Horizons-Bridging Cultures, Languages, and Domains in Video Comprehension](https://mlsanity.com/paper/2504.17821/) (2025)
- [Factorized Learning for Temporally Grounded Video-Language Models](https://mlsanity.com/paper/2512.24097/) (2025)
- [TEMPLE: Incentivizing Temporal Understanding of Video Large Language Models via Progressive Pre-SFT Alignment](https://mlsanity.com/paper/2503.16929/) (2025)
- [SynPO: Synergizing Descriptiveness and Preference Optimization for Video Detailed Captioning](https://mlsanity.com/paper/2506.00835/) (2025)
- [AdaptToken: Entropy-based Adaptive Token Selection for MLLM Long Video Understanding](https://mlsanity.com/paper/2603.28696/) (2026)
- [Reinforcing Consistency in Video MLLMs with Structured Rewards](https://mlsanity.com/paper/2604.01460/) (2026)
- [LeanPO: Lean Preference Optimization for Likelihood Alignment in Video-LLMs](https://mlsanity.com/paper/2506.05260/) (2025)
- [How Important are Videos for Training Video LLMs?](https://mlsanity.com/paper/2506.06928/) (2025)
- [Benchmarking the Trustworthiness in Multimodal LLMs for Video Understanding](https://mlsanity.com/paper/2506.12336/) (2025)
- [LongVPO: From Anchored Cues to Self-Reasoning for Long-Form Video Preference Optimization](https://mlsanity.com/paper/2602.02341/) (2026)
- [Towards Temporal Compositional Reasoning in Long-Form Sports Videos](https://mlsanity.com/paper/2604.22226/) (2026)
- [Oracle-RLAIF: An Improved Fine-Tuning Framework for Multi-modal Video Models using Reinforcement Learning from Ranking Feedback](https://mlsanity.com/paper/2510.02561/) (2025)
- [Bridging VideoQA and Video-Guided Agentic Tasks via Generalized Keyframe Extraction](https://mlsanity.com/paper/2606.29445/) (2026)
- [VADER: Adaptive Debiasing for Hallucination Mitigation in Video Large Language Models](https://mlsanity.com/paper/2608.08622/) (2026)
- [Multi-Agent Self-Improving Reinforcement Learning for Video Reasoning](https://mlsanity.com/paper/2608.28675/) (2026)

## Abstract (arXiv)

Despite significant advancements in video large multimodal models (video-LMMs), achieving effective temporal grounding in long-form videos remains a challenge for existing models. To address this limitation, we propose Temporal Preference Optimization (TPO), a novel post-training framework designed to enhance the temporal grounding capabilities of video-LMMs through preference learning. TPO adopts a self-training approach that enables models to differentiate between well-grounded and less accurate temporal responses by leveraging curated preference datasets at two granularities: localized temporal grounding, which focuses on specific video segments, and comprehensive temporal grounding, which captures extended temporal dependencies across entire video sequences. By optimizing on these preference datasets, TPO significantly enhances temporal understanding while reducing reliance on manually annotated data. Extensive experiments on three long-form video understanding benchmarks--LongVideoBench, MLVU, and Video-MME--demonstrate the effectiveness of TPO across two state-of-the-art video-LMMs. Notably, LLaVA-Video-TPO establishes itself as the leading 7B model on the Video-MME benchmark, underscoring the potential of TPO as a scalable and efficient solution for advancing temporal reasoning in long-form video understanding. Project page: https://ruili33.github.io/tpo_website.