DELTAVID: Enhancing Fine-Grained Spatiotemporal Perception with Cross-Video Differences
On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs
Abstract (arXiv)
Video multimodal large language models have made strong progress on open-ended video understanding, but they still lack precise local spatiotemporal perception. When two videos share almost the same global semantics and differ only in a short time span or a small region, current models often fail to find the change and provide reliable evidence. We propose DELTAVID, a verifiable proxy-task framework that enhances fine-grained spatiotemporal perception with cross-video differences. The key idea is to turn cross-video spot-the-difference into a trainable perception signal, where a model identifies local changes, judges temporal boundaries, and organizes spatial evidence by comparing similar videos. To make this signal scalable to train and reliable to evaluate, we further introduce DELTAVID-10K and DELTAVID-Bench, which convert controllable local differences in real videos into evidence-labeled training and test samples. Experiments show that DELTAVID substantially improves performance on cross-video difference understanding and transfers the learned local evidence ability to general video understanding benchmarks, including MMVU, MLVU, Video-MME, VideoHolmes, VideoMMMU, LVBench, TempCompass, and LongVideoBench. These results show that cross-video differences are not only an effective way to diagnose fine-grained perception failures, but also a scalable proxy supervision that moves Video MLLMs from coarse semantic understanding toward fine-grained spatiotemporal evidence reasoning.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 41 of its 47 references that are in the map. First posted on arXiv 26 Jun 2026.
Builds on
41 of 47 references.
Closest ancestors · 10
Same direction, newest first
- 2025VideoZoomer: Reinforcement-Learned Temporal Focusing for Long Video Reasoning26 Dec
- 2025TempR1: Improving Temporal Understanding of MLLMs via Temporal-Aware Multi-Task Reinforcement Learning3 Dec
- 2025Video-R1: Reinforcing Video Reasoning in MLLMs27 Mar
- 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding22 Jul
- 2024LVBench: An Extreme Long Video Understanding Benchmark12 Jun
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis31 May
- 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding4 Dec
- 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models28 Nov
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projection16 Nov
- 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding5 Jun
Other references · 31
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2023Visual Instruction Tuningcited by 7,792
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
- 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
- 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
- 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
19 more
- 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
- 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
- 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
- 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
- 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
- 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
- 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
- 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
- 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 364
- 2024TempCompass: Do Video LLMs Really Understand Videos?cited by 352
- 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 310
- 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 296
- 2025Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videoscited by 225
- 2024World Model on Million-Length Video And Language With Blockwise RingAttentioncited by 196
- 2018Learning to Describe Differences Between Pairs of Similar Imagescited by 114
- 2025Video-Holmes: Can MLLM Think Like Holmes for Complex Video Reasoning?cited by 107
- 2025Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Modelscited by 92
- 2025Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzlescited by 36
- 2025ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMscited by 25
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Models
- 2025Know-Show: Benchmarking Video-Language Models on Spatio-Temporal Grounded Reasoning
- 2024MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding
- 2026Temporal Gains, Spatial Costs: Revisiting Video Fine-Tuning in Multimodal Large Language Models
- 2025Improving LLM Video Understanding with 16 Frames Per Second
- 2026VISTA: Video Interaction Spatio-Temporal Analysis Benchmark
- 2025VideoPerceiver: Enhancing Fine-Grained Temporal Perception in Video Multimodal Large Language Models
- 2025CrossVid: A Comprehensive Benchmark for Evaluating Cross-Video Reasoning in Multimodal Large Language Models