mlsanityData to 7 Oct 2026

Egocentric Bias in Vision-Language Models

arXiv 2602.15892 ↗First posted 10 Feb 2026cs.CV cs.AIcited by 3

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Visual perspective taking--inferring how the world appears from another's viewpoint--is foundational to social cognition. We introduce FlipSet, a diagnostic benchmark for Level-2 visual perspective taking (L2 VPT) in vision-language models. The task requires simulating 180-degree rotations of 2D character strings from another agent's perspective, isolating spatial transformation from 3D scene complexity. Evaluating 103 VLMs reveals systematic egocentric bias: the vast majority perform below chance, with roughly three-quarters of errors reproducing the camera viewpoint. Control experiments expose a compositional deficit--models achieve high theory-of-mind accuracy and above-chance mental rotation in isolation, yet fail catastrophically when integration is required. This dissociation indicates that current VLMs lack the mechanisms needed to bind social awareness to spatial operations, suggesting fundamental limitations in model-based spatial reasoning. FlipSet provides a cognitively grounded testbed for diagnosing perspective-taking capabilities in multimodal systems.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 21 of its 40 references that are in the map. First posted on arXiv 10 Feb 2026.

Builds on

21 of 40 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Do Vision-Language Models Have Internal World Models? Towards an Atomic Evaluation27 Jun
  2. 2024Do Vision-Language Models Represent Space and How? Evaluating Spatial Frame of Reference Under Ambiguities22 Oct
  3. 2024Improve Vision Language Model Chain-of-thought Reasoning21 Oct
  4. 2024Probing Mechanical Reasoning in Large Vision Language Models1 Oct

Other references · 17

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  4. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  5. 2024VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMscited by 749
  6. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 553
  7. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 446
  8. 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 329
  9. 2024Vision-Language Models Can Self-Improve Reasoning via Reflectioncited by 66
  10. 2024Can Language Models Serve as Text-Based World Simulators?cited by 50
  11. 2024Core Knowledge Deficits in Multi-Modal Language Modelscited by 38
  12. 2024Exploring Perceptual Limitation of Multimodal Large Language Modelscited by 24
5 more
  1. 2024Vision Language Models See What You Want but not What You Seecited by 10
  2. 2025VIDEOP2R: Video Understanding from Perception to Reasoningcited by 8
  3. 2025Probing Perceptual Constancy in Large Vision-Language Modelscited by 7
  4. 2025The Philosophical Foundations of Growing AI Like A Childcited by 6
  5. 2025Rethinking the Simulation vs. Rendering Dichotomy: No Free Lunch in Spatial World Modellingcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026DRIVESPATIAL: A Benchmark for Spatiotemporal Intelligence in VLMs for Autonomous Drivingcited by 2
  2. 2026Beyond Localization: A Comprehensive Diagnosis of Perspective-Conditioned Spatial Reasoning in MLLMs from Omnidirectional Imagescited by 0
  3. 2026Restrict, Don't Retrain: Inference-Time VLM Guidance for Zero-Shot Aerial Segmentationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025SpinBench: Perspective and Rotation as a Lens on Spatial Reasoning in VLMs
  2. 2025Perspective-Aware Reasoning in Vision-Language Models via Mental Imagery Simulation
  3. 2026Cognitively-Inspired Tokens Overcome Egocentric Bias in Multimodal Models
  4. 2025Beyond Recognition: Evaluating Visual Perspective Taking in Vision Language Models
  5. 2026Ego to World: Collaborative Spatial Reasoning in Embodied Systems via Reinforcement Learning
  6. 2025MindCube: Spatial Mental Modeling from Limited Views
  7. 2025ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models
  8. 2025Reasoning Path and Latent State Analysis for Multi-view Visual Spatial Reasoning: A Cognitive Science Perspective