Image Difference Grounding with Natural Language
Not on the map. Why?
Abstract (arXiv)
Visual grounding (VG) typically focuses on locating regions of interest within an image using natural language, and most existing VG methods are limited to single-image interpretations. This limits their applicability in real-world scenarios like automatic surveillance, where detecting subtle but meaningful visual differences across multiple images is crucial. Besides, previous work on image difference understanding (IDU) has either focused on detecting all change regions without cross-modal text guidance, or on providing coarse-grained descriptions of differences. Therefore, to push towards finer-grained vision-language perception, we propose Image Difference Grounding (IDG), a task designed to precisely localize visual differences based on user instructions. We introduce DiffGround, a large-scale and high-quality dataset for IDG, containing image pairs with diverse visual variations along with instructions querying fine-grained differences. Besides, we present a baseline model for IDG, DiffTracker, which effectively integrates feature differential enhancement and common suppression to precisely locate differences. Experiments on the DiffGround dataset highlight the importance of our IDG dataset in enabling finer-grained IDU. To foster future research, both DiffGround data and DiffTracker model will be publicly released.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 56 of its 71 references that are in the map. First posted on arXiv 2 Apr 2025.
Builds on
56 of 71 references.
References · 56
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2020End-to-End Object Detection with Transformerscited by 7,414
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
- 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
- 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
- 2024Qwen2 Technical Reportcited by 2,120
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
38 more
- 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,368
- 2016Modeling Context in Referring Expressionscited by 1,063
- 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
- 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
- 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
- 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
- 2022A Generalist Agentcited by 808
- 2023LISA: Reasoning Segmentation via Large Language Modelcited by 776
- 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
- 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 509
- 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 455
- 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 439
- 2023An Embodied Generalist Agent in 3D Worldcited by 365
- 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
- 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 348
- 2022Generalized Decoding for Pixel, Image, and Languagecited by 294
- 2021TransVG: End-to-End Visual Grounding with Transformerscited by 271
- 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 268
- 2019Cross-Modal Self-Attention Network for Referring Image Segmentationcited by 263
- 2016Segmentation from Natural Language Expressionscited by 257
- 2016Modeling Context Between Objects for Referring Expression Understandingcited by 251
- 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 240
- 2023Inpaint Anything: Segment Anything Meets Image Inpaintingcited by 233
- 2023Physically Grounded Vision-Language Models for Robotic Manipulationcited by 165
- 2023GSVA: Generalized Segmentation via Multimodal Large Language Modelscited by 154
- 2022SeqTR: A Simple yet Universal Network for Visual Groundingcited by 139
- 2024ChangeMamba: Remote Sensing Change Detection With Spatiotemporal State Space Modelcited by 120
- 2018Learning to Describe Differences Between Pairs of Similar Imagescited by 114
- 2022Changer: Feature Interaction is What You Need for Change Detectioncited by 79
- 2018Real-Time Referring Expression Comprehension by Single-Stage Grounding Networkcited by 56
- 2019Expressing Visual Relationships via Languagecited by 49
- 2023A New Learning Paradigm for Foundation Model-based Remote Sensing Change Detectioncited by 47
- 2022Joint Spatio-Temporal Modeling for the Semantic Change Detection in Remote Sensing Imagescited by 43
- 2022TransVG++: End-to-End Visual Grounding with Language Conditioned Vision Transformercited by 42
- 2022Image Difference Captioning with Pre-training and Contrastive Learningcited by 39
- 2019Neural Naturalist: Generating Fine-Grained Image Comparisonscited by 36
- 2020Finding It at Another Side: A Viewpoint-Adapted Matching Encoder for Change Captioningcited by 26
- 2022CLIP4IDC: CLIP for Image Difference Captioningcited by 25
50 most cited of 56 All 56 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.