# Spot-the-shift: Evaluating Grounded Image Difference Captioning of Long-term Changes

arXiv: https://arxiv.org/abs/2609.10356 · HTML: https://mlsanity.com/paper/2609.10356/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 35 of its 52 references that are in the map.
- First posted on arXiv 9 Sep 2026.

## Builds on (most cited first, 30 of 35)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [Evaluating Object Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2305.10355/) (2023)
- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025)
- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs](https://mlsanity.com/paper/2406.16860/) (2024)
- [LISA: Reasoning Segmentation via Large Language Model](https://mlsanity.com/paper/2308.00692/) (2023)
- [BLINK: Multimodal Large Language Models Can See but Not Perceive](https://mlsanity.com/paper/2404.12390/) (2024)
- [MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding](https://mlsanity.com/paper/2406.09411/) (2024)
- [Osprey: Pixel Understanding with Visual Instruction Tuning](https://mlsanity.com/paper/2312.10032/) (2023)
- [Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding](https://mlsanity.com/paper/2601.10611/) (2026)
- [Learning to Describe Differences Between Pairs of Similar Images](https://mlsanity.com/paper/1808.10584/) (2018)
- [UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learning](https://mlsanity.com/paper/2505.14231/) (2025)
- [Neural Map Prior for Autonomous Driving](https://mlsanity.com/paper/2304.08481/) (2023)
- [ALOHa: A New Measure for Hallucination in Captioning Models](https://mlsanity.com/paper/2404.02904/) (2024)
- [Weakly Supervised Silhouette-based Semantic Scene Change Detection](https://mlsanity.com/paper/1811.11985/) (2018)
- [CLAIR: Evaluating Image Captions with Large Language Models](https://mlsanity.com/paper/2310.12971/) (2023)
- [THRONE: An Object-based Hallucination Benchmark for the Free-form Generations of Large Vision-Language Models](https://mlsanity.com/paper/2405.05256/) (2024)
- [Neptune: The Long Orbit to Benchmarking Long Video Understanding](https://mlsanity.com/paper/2412.09582/) (2024)
- [G-VEval: A Versatile Metric for Evaluating Image and Video Captions Using GPT-4o](https://mlsanity.com/paper/2412.13647/) (2024)
- [Unsupervised Change Detection Based on Image Reconstruction Loss](https://mlsanity.com/paper/2204.01200/) (2022)
- [Self-Pair: Synthesizing Changes from Single Source for Object Change Detection in Remote Sensing Imagery](https://mlsanity.com/paper/2212.10236/) (2022)
- [Towards Generalizable Scene Change Detection](https://mlsanity.com/paper/2409.06214/) (2024)
- [MapFormer: Boosting Change Detection by Using Pre-change Information](https://mlsanity.com/paper/2303.17859/) (2023)
- [ZeroSCD: Zero-Shot Street Scene Change Detection](https://mlsanity.com/paper/2409.15255/) (2024)
- [OmniDiff: A Comprehensive Benchmark for Fine-grained Image Difference Captioning](https://mlsanity.com/paper/2503.11093/) (2025)

## Abstract (arXiv)

Long-term change understanding from images of the same place revisited over time is a challenging task with applications in map maintenance and urban infrastructure monitoring. Prior work addresses it either through pixel-level prediction or difference captioning, neither of which is sufficient to reliably measure how well models detect and describe such changes. We introduce SPOT-THE-SHIFT, a human-verified benchmark for grounded image difference captioning of long-term changes in real-world driving scenes. Our benchmark provides natural language captions and spatial masks for structural changes across each image pair. We further propose an evaluation protocol that reliably assesses models' captioning ability, validated through human studies. Benchmarking state-of-the-art MLLMs, we find that models struggle with the fine-grained multi-image spatial capability required for this task. Finally, we develop a synthetic data generation pipeline that improves an off-the-shelf MLLM without sacrificing general capabilities.