mlsanityData to 14 Oct 2026

Seeing Depth but Choosing Cues in Vision-Language Models

arXiv 2610.17976 ↗First posted 13 Oct 2026cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning also fits: Embodied and 3D vision-language models

Abstract (arXiv)

Do vision-language models use depth information to judge which object is closer, or rely on image-plane cues? We investigate this question through controlled depth--cue conflicts, representation analysis, and interventions. Among the many image-plane cues relevant to depth perception, we focus on apparent size and vertical position. We compare depth judgments under cue-consistent and cue-conflict conditions in both rendered scenes and real-world photographs. Across model families and scales, depth judgments systematically shift toward misleading cues. Yet the correct depth order remains decodable from the queried objects' visual representations, including when the model answers incorrectly. Image and category controls support interpreting this signal as image-dependent depth information beyond the two target cues. Strengthening the model's own depth readout through object-level write-back can correct conflict decisions without updating model parameters or using ground-truth depth annotations. Interventions show that the effect of this correction depends on the direction and target tokens of the written signal, while its strength varies across architectures and domains. Together, these findings reveal a mismatch between a model's ability to represent depth and use this information when deciding the answer. VLMs can retain useful depth information while allowing competing image-plane cues to exert greater influence on their answers.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 17 of its 19 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

17 of 19 references.

References · 17

  1. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  2. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  3. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  4. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  5. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  6. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 706
  7. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
  8. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 688
  9. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 609
  10. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 550
  11. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 376
  12. 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Modelscited by 311
5 more
  1. 2023LEACE: Perfect linear concept erasure in closed formcited by 267
  2. 2026Gemma 4 Technical Reportcited by 259
  3. 2026Linear Mechanisms for Spatiotemporal Reasoning in Vision Language Modelscited by 14
  4. 2026Why Far Looks Up: Probing Spatial Representation in Vision-Language Modelscited by 3
  5. 2026Decodable Is Not Grounded: A Vision-Ablation Arbiter for VLM Spatial Reasoningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Disentangling Pictorial Cue Understanding from Language Bias in VLMs via Depth Ordering Task
  2. 2026When Depth Is Better Told Than Shown: Depth-Ordinal Prompting for Vision-Language Spatial Reasoning
  3. 2026Do Vision-Language Models Understand 3D Scenes or Just Catalogue Objects?
  4. 2026Anchored, Not Graded: Vision-Language Models Fail at Slant-from-Texture Perception
  5. 2025Mixed Signals: Decoding VLMs' Reasoning and Underlying Bias in Vision-Language Conflict
  6. 2024Understanding Depth and Height Perception in Large Visual-Language Models
  7. 2026Seeing without Looking: Do Vision-Language Benchmarks Really Test Vision?
  8. 2026Contextual inference from single objects in Vision-Language models