mlsanityData to 7 Oct 2026

Learning to Describe Differences Between Pairs of Similar Images

arXiv 1808.10584 ↗First posted 31 Aug 2018cs.CL cs.CVcited by 114

Not on the map. Why?

Cited by 44 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this paper, we introduce the task of automatically generating text to describe the differences between two similar images. We collect a new dataset by crowd-sourcing difference descriptions for pairs of image frames extracted from video-surveillance footage. Annotators were asked to succinctly describe all the differences in a short paragraph. As a result, our novel dataset provides an opportunity to explore models that align language and vision, and capture visual salience. The dataset may also be a useful benchmark for coherent multi-sentence generation. We perform a firstpass visual analysis that exposes clusters of differing pixels as a proxy for object-level differences. We propose a model that captures visual salience by using a latent variable to align clusters of differing pixels with output sentences. We find that, for both single-sentence generation and as well as multi-sentence generation, the proposed model outperforms the models that use attention alone.

Key facts. Cited by 114 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Aug 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 10; 2023: 12; 2024: 33; 2025: 27; 2026: 25.

114 papers.

Sort:

By area:

  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  2. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 548
  3. 2022SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentationcited by 397
  4. 2024What matters when building vision-language models?cited by 321
  5. 2023MIMIC-IT: Multi-Modal In-Context Instruction Tuningcited by 264
  6. 2021Image Retrieval on Real-life Images with Pre-trained Vision-and-Language Modelscited by 244
  7. 2024MANTIS: Interleaved Multi-Image Instruction Tuningcited by 212
  8. 2024Building and better understanding vision-language models: insights and future directionscited by 169
  9. 2021From Show to Tell: A Survey on Deep Learning-based Image Captioningcited by 137
  10. 2023VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Usecited by 100
  11. 2024MileBench: Benchmarking MLLMs in Long Contextcited by 86
  12. 2023Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructionscited by 76
38 more
  1. 2024MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuningcited by 75
  2. 2025Eagle 2: Building Post-Training Data Strategies from Scratch for Frontier Vision-Language Modelscited by 69
  3. 2025A Vision-Language-Action-Critic Model for Robotic Real-World Reinforcement Learningcited by 66
  4. 2024MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Modelscited by 61
  5. 2025FineVision: Open Data Is All You Needcited by 51
  6. 2021Describing and Localizing Multiple Changes with Transformerscited by 47
  7. 2023Changes to Captions: An Attentive Network for Remote Sensing Change Captioningcited by 41
  8. 2023Expert Knowledge-Aware Image Difference Graph Representation Learning for Difference-Aware Medical Visual Question Answeringcited by 41
  9. 2022Image Difference Captioning with Pre-training and Contrastive Learningcited by 39
  10. 2023Visual Instruction Tuning with Polite Flamingocited by 38
  11. 2022Image Retrieval from Contextual Descriptionscited by 26
  12. 2022CLIP4IDC: CLIP for Image Difference Captioningcited by 25
  13. 2024VisMin: Visual Minimal-Change Understandingcited by 23
  14. 2025Migician: Revealing the Magic of Free-Form Multi-Image Grounding in Multimodal Large Language Modelscited by 22
  15. 2023Self-supervised Cross-view Representation Reconstruction for Change Captioningcited by 20
  16. 2026LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decodingcited by 20
  17. 2022GEB+: A Benchmark for Generic Event Boundary Captioning, Grounding and Retrievalcited by 19
  18. 2023The Change You Want to See (Now in 3D)cited by 17
  19. 2024Img-Diff: Contrastive Data Synthesis for Multimodal Large Language Modelscited by 17
  20. 2025A Comprehensive Survey on Composed Image Retrievalcited by 17
  21. 2023Neighborhood Contrastive Transformer for Change Captioningcited by 16
  22. 2024Enhancing Perception of Key Changes in Remote Sensing Image Change Captioningcited by 16
  23. 2024OneDiff: A Generalist Model for Image Difference Captioningcited by 15
  24. 2025NVIDIA Nemotron Nano V2 VLcited by 15
  25. 2025Video Action Differencingcited by 14
  26. 2024CCExpert: Advancing MLLM Capability in Remote Sensing Change Captioning with Difference-Aware Integration and a Foundational Datasetcited by 13
  27. 2021FixMyPose: Pose Correctional Captioning and Retrievalcited by 12
  28. 2021R3Net:Relation-embedded Representation Reconstruction Network for Change Captioningcited by 12
  29. 2024VIXEN: Visual Text Comparison Network for Image Difference Captioningcited by 12
  30. 2024Semantic Alignment for Multimodal Large Language Modelscited by 12
  31. 2024Multi-View Pose-Agnostic Change Localization with Zero Labelscited by 12
  32. 2025OmniDiff: A Comprehensive Benchmark for Fine-grained Image Difference Captioningcited by 12
  33. 2025MadaKV: Adaptive Modality-Perception KV Cache Eviction for Efficient Multimodal Long-Context Inferencecited by 12
  34. 2021Truth-Conditional Captioning of Time Series Datacited by 11
  35. 2024Step Differences in Instructional Videocited by 11
  36. 2024Cross-Self KV Cache Pruning for Efficient Vision-Language Inferencecited by 11
  37. 2022Reasoning about Actions over Visual and Linguistic Modalities: A Surveycited by 10
  38. 2024Context-aware Difference Distilling for Multi-change Captioningcited by 10

50 most cited of 114 All 114 (JSON)