# Temporal Contrastive Learning for Video Temporal Reasoning in Large Vision-Language Models

arXiv: https://arxiv.org/abs/2412.11391 · HTML: https://mlsanity.com/paper/2412.11391/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 14 of its 27 references that are in the map.
- First posted on arXiv 16 Dec 2024.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [Video understanding with MLLMs](https://mlsanity.com/mllm/video-understanding-with-mllms/)

## Builds on (most cited first, 14 of 14)

- [MoE-LLaVA: Mixture of Experts for Large Vision-Language Models](https://mlsanity.com/paper/2401.15947/) (2024)
- [An Introduction to Vision-Language Modeling](https://mlsanity.com/paper/2405.17247/) (2024)
- [Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning](https://mlsanity.com/paper/2405.10292/) (2024)
- [TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Models](https://mlsanity.com/paper/2410.10818/) (2024)
- [CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignment](https://mlsanity.com/paper/2209.06430/) (2022)
- [Thread of Thought Unraveling Chaotic Contexts](https://mlsanity.com/paper/2311.08734/) (2023)
- [TimeMarker: A Versatile Video-LLM for Long and Short Video Understanding with Superior Temporal Localization Ability](https://mlsanity.com/paper/2411.18211/) (2024)
- [Rethinking Visual Dependency in Long-Context Reasoning for Large Vision-Language Models](https://mlsanity.com/paper/2410.19732/) (2024)
- [Temporal Reasoning Transfer from Text to Video](https://mlsanity.com/paper/2410.06166/) (2024)
- [Temporal Sentence Grounding in Videos: A Survey and Future Directions](https://mlsanity.com/paper/2201.08071/) (2022)
- [TextHawk2: A Large Vision-Language Model Excels in Bilingual OCR and Grounding with 16x Fewer Tokens](https://mlsanity.com/paper/2410.05261/) (2024)
- [RelationVLM: Making Large Vision-Language Models Understand Visual Relations](https://mlsanity.com/paper/2403.12801/) (2024)
- [Towards Effective Time-Aware Language Representation: Exploring Enhanced Temporal Understanding in Language Models](https://mlsanity.com/paper/2406.01863/) (2024)
- [Temporal2Seq: A Unified Framework for Temporal Video Understanding Tasks](https://mlsanity.com/paper/2409.18478/) (2024)

## Built on by (most cited first, 1 of 1)

- [Event-Causal RAG: A Retrieval-Augmented Generation Framework for Long Video Reasoning in Complex Scenarios](https://mlsanity.com/paper/2605.06185/) (2026)

## Abstract (arXiv)

Temporal reasoning is a critical challenge in video-language understanding, as it requires models to align semantic concepts consistently across time. While existing large vision-language models (LVLMs) and large language models (LLMs) excel at static tasks, they struggle to capture dynamic interactions and temporal dependencies in video sequences. In this work, we propose Temporal Semantic Alignment via Dynamic Prompting (TSADP), a novel framework that enhances temporal reasoning capabilities through dynamic task-specific prompts and temporal contrastive learning. TSADP leverages a Dynamic Prompt Generator (DPG) to encode fine-grained temporal relationships and a Temporal Contrastive Loss (TCL) to align visual and textual embeddings across time. We evaluate our method on the VidSitu dataset, augmented with enriched temporal annotations, and demonstrate significant improvements over state-of-the-art models in tasks such as Intra-Video Entity Association, Temporal Relationship Understanding, and Chronology Prediction. Human evaluations further confirm TSADP's ability to generate coherent and semantically accurate descriptions. Our analysis highlights the robustness, efficiency, and practical utility of TSADP, making it a step forward in the field of video-language understanding.