SEER: A Self-Grounded Evidence Interface for Controlled Spatial Relation Classification
Not on the map. Why?
Abstract (arXiv)
Spatial relation questions require a model to identify the queried subject and object before comparing their layout. Yet a VLM can recognize both entities and still answer from the wrong instance or an ambiguous global view. We ask whether making query-specific evidence explicit can mitigate this failure and propose SEER (Self-grounded Evidence for Entity-Relation Reasoning), a training-free inference-time evidence interface for frozen VLMs. SEER hides candidate relations during pair localization, constructs a query-specific view with explicit subject/object roles, and retains the full image and sparse box geometry as complementary evidence. For relation-choice protocols with exact inverse support, an optional refinement swaps the entity roles and changes the forward decision only when exactly one visual state obeys the corresponding inverse relation. On an image-disjoint GQA-Train900 test frozen before model scoring, SEER pools to +3.94 [2.17,5.72] over Full; the gain remains positive under label-independent grounding-order counterbalancing and on the 535 rows whose entity names are unique. The unchanged protocol yields +4.35 to +11.79 on all 2,434 filtered EmbSpatial pair-relation questions across three models. Matched controls separate local refocus from role-explicit conditioning. These results establish query-specific evidence construction as the principal intervention, with reciprocal consistency as a smaller protocol-specific refinement.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 26 of its 37 references that are in the map. First posted on arXiv 4 Aug 2026.
Builds on
26 of 37 references.
References · 26
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
- 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
- 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
- 2015VQA: Visual Question Answeringcited by 2,506
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
- 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
- 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
- 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
14 more
- 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 706
- 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 610
- 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 529
- 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 509
- 2025R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalizationcited by 370
- 2022Simple Open-Vocabulary Object Detection with Vision Transformerscited by 305
- 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoningcited by 254
- 2023SugarCrepe: Fixing Hackable Benchmarks for Vision-Language Compositionalitycited by 243
- 2021VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomenacited by 135
- 2019SpatialSense: An Adversarially Crowdsourced Benchmark for Spatial Relation Recognitioncited by 55
- 2024GSR-BENCH: A Benchmark for Grounded Spatial Reasoning Evaluation via Multimodal LLMscited by 33
- 2024MM-R3: On (In-)Consistency of Vision-Language Models (VLMs)cited by 10
- 2025Reallocating Attention Across Layers to Reduce Multimodal Hallucinationcited by 4
- 2026Graph-of-Mark: Promote Spatial Reasoning in Multimodal Language Models with Graph-Based Visual Promptingcited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026RECAP: Relation Evidence Calibration for Detecting Spatial Relation Hallucinations in Vision-Language Models
- 2025Understanding Space Is Rocket Science -- Only Top Reasoning Models Can Solve Spatial Understanding Tasks
- 2026Grounding Isn't Knowing: Do VLMs Need Object Localization for Spatial Reasoning?
- 2024The All-Seeing Project V2: Towards General Relation Comprehension of the Open World
- 2026Who Is Left of Whom? Tracing Spatial Evidence and Role Binding in Relative-Position Reasoning
- 2024Grounding Spatial Relations in Text-Only Language Models
- 2026SpatialChain: A Benchmark for Auditing Spatial Reasoning Faithfulness in VLMs
- 2025SpaRE: Enhancing Spatial Reasoning in Vision-Language Models with Synthetic Data