mlsanityData to 7 Oct 2026

Semantic Richness or Geometric Reasoning? The Fragility of VLM's Visual Invariance

arXiv 2604.01848 ↗First posted 2 Apr 2026cs.CVcited by 0

On the map: LLM reasoning → Spatial and embodied reasoning

also in 1 more
Abstract (arXiv)

This work investigates the fundamental fragility of state-of-the-art Vision-Language Models (VLMs) under basic geometric transformations. While modern VLMs excel at semantic tasks such as recognizing objects in canonical orientations and describing complex scenes, they exhibit systematic failures at a more fundamental level: lack of robust spatial invariance and equivariance required to reliably determine object identity under simple rotations, scaling, and identity transformations. We demonstrate this limitation through a systematic evaluation across diverse visual domains, including symbolic sketches, natural photographs, and abstract art. Performance drops sharply as semantic content becomes sparse, and this behavior is observed across architectures, model capacities, and prompting strategies. Overall, our results reveal a systematic gap between semantic understanding and spatial reasoning in current VLMs, highlighting the need for stronger geometric grounding in future multimodal systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 18 of its 36 references that are in the map. First posted on arXiv 2 Apr 2026.

Builds on

18 of 36 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Large Vision Models Can Solve Mental Rotation Problems18 Sep

Other references · 17

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 2024Qwen2.5 Technical Reportcited by 4,713
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  5. 2025Qwen3-VL Technical Reportcited by 2,534
  6. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  7. 2018On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groupscited by 296
  8. 2024Understanding the Limits of Vision Language Models Through the Lens of the Binding Problemcited by 71
  9. 2024Does Spatial Cognition Emerge in Frontier Models?cited by 65
  10. 2025Vision Language Models are Biasedcited by 63
  11. 2026BabyVision: Visual Reasoning Beyond Languagecited by 39
  12. 2022PACS: A Dataset for Physical Audiovisual CommonSense Reasoningcited by 17
5 more
  1. 2024Bongard in Wonderland: Visual Puzzles that Still Make AI Go Mad?cited by 16
  2. 2023Reviving Shift Equivariance in Vision Transformerscited by 9
  3. 2025Position: Reasoning After Perception Means Reasoning Without Visioncited by 8
  4. 2025Visually Prompted Benchmarks Are Surprisingly Fragilecited by 6
  5. 2025RotBench: Evaluating Multimodal Large Language Models on Identifying Image Rotationcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Render to Reason: Novel-View Semantic Prediction Improves Spatial Understanding in VLMs
  2. 2026Spatial Reasoning is Not a Free Lunch: A Controlled Study on LLaVA
  3. 2025Vision language models are unreliable at trivial spatial cognition
  4. 2025Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models
  5. 2025Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas
  6. 2025SpinBench: Perspective and Rotation as a Lens on Spatial Reasoning in VLMs
  7. 2026Investigating Relational Reasoning in VLMs
  8. 2026GeoWorld-VLM: Geometry from World Models for Vision-Language Models