mlsanityData to 7 Oct 2026

DELTAVID: Enhancing Fine-Grained Spatiotemporal Perception with Cross-Video Differences

arXiv 2607.02551 ↗First posted 26 Jun 2026cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

Video multimodal large language models have made strong progress on open-ended video understanding, but they still lack precise local spatiotemporal perception. When two videos share almost the same global semantics and differ only in a short time span or a small region, current models often fail to find the change and provide reliable evidence. We propose DELTAVID, a verifiable proxy-task framework that enhances fine-grained spatiotemporal perception with cross-video differences. The key idea is to turn cross-video spot-the-difference into a trainable perception signal, where a model identifies local changes, judges temporal boundaries, and organizes spatial evidence by comparing similar videos. To make this signal scalable to train and reliable to evaluate, we further introduce DELTAVID-10K and DELTAVID-Bench, which convert controllable local differences in real videos into evidence-labeled training and test samples. Experiments show that DELTAVID substantially improves performance on cross-video difference understanding and transfers the learned local evidence ability to general video understanding benchmarks, including MMVU, MLVU, Video-MME, VideoHolmes, VideoMMMU, LVBench, TempCompass, and LongVideoBench. These results show that cross-video differences are not only an effective way to diagnose fine-grained perception failures, but also a scalable proxy supervision that moves Video MLLMs from coarse semantic understanding toward fine-grained spatiotemporal evidence reasoning.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 41 of its 47 references that are in the map. First posted on arXiv 26 Jun 2026.

Builds on

41 of 47 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025VideoZoomer: Reinforcement-Learned Temporal Focusing for Long Video Reasoning26 Dec
  2. 2025TempR1: Improving Temporal Understanding of MLLMs via Temporal-Aware Multi-Task Reinforcement Learning3 Dec
  3. 2025Video-R1: Reinforcing Video Reasoning in MLLMs27 Mar
  4. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding22 Jul
  5. 2024LVBench: An Extreme Long Video Understanding Benchmark12 Jun
  6. 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis31 May
  7. 2023TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding4 Dec
  8. 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models28 Nov
  9. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projection16 Nov
  10. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding5 Jun

Other references · 31

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  5. 2023Visual Instruction Tuningcited by 7,792
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  7. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  8. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  9. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  10. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
  11. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  12. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
19 more
  1. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  2. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  3. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  4. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
  5. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
  6. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
  7. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
  8. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
  9. 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 364
  10. 2024TempCompass: Do Video LLMs Really Understand Videos?cited by 352
  11. 2024InternVideo2: Scaling Foundation Models for Multimodal Video Understandingcited by 310
  12. 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 296
  13. 2025Video-MMMU: Evaluating Knowledge Acquisition from Multi-Discipline Professional Videoscited by 225
  14. 2024World Model on Million-Length Video And Language With Blockwise RingAttentioncited by 196
  15. 2018Learning to Describe Differences Between Pairs of Similar Imagescited by 114
  16. 2025Video-Holmes: Can MLLM Think Like Holmes for Complex Video Reasoning?cited by 107
  17. 2025Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Modelscited by 92
  18. 2025Jigsaw-R1: A Study of Rule-based Visual Reinforcement Learning with Jigsaw Puzzlescited by 36
  19. 2025ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMscited by 25

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Models
  2. 2025Know-Show: Benchmarking Video-Language Models on Spatio-Temporal Grounded Reasoning
  3. 2024MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding
  4. 2026Temporal Gains, Spatial Costs: Revisiting Video Fine-Tuning in Multimodal Large Language Models
  5. 2025Improving LLM Video Understanding with 16 Frames Per Second
  6. 2026VISTA: Video Interaction Spatio-Temporal Analysis Benchmark
  7. 2025VideoPerceiver: Enhancing Fine-Grained Temporal Perception in Video Multimodal Large Language Models
  8. 2025CrossVid: A Comprehensive Benchmark for Evaluating Cross-Video Reasoning in Multimodal Large Language Models