# Failures in Perspective-taking of Multimodal AI Systems

arXiv: https://arxiv.org/abs/2409.13929 · HTML: https://mlsanity.com/paper/2409.13929/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 4 references that are in the map.
- First posted on arXiv 20 Sep 2024.

## Builds on (most cited first, 1 of 1)

- [BLINK: Multimodal Large Language Models Can See but Not Perceive](https://mlsanity.com/paper/2404.12390/) (2024)

## Built on by (most cited first, 6 of 6)

- [PerspAct: Enhancing LLM Situated Collaboration Skills through Perspective Taking and Active Vision](https://mlsanity.com/paper/2511.08098/) (2025)
- [Visuospatial Perspective Taking in Multimodal Language Models](https://mlsanity.com/paper/2603.23510/) (2026)
- [Are Vision Language Models Cross-Cultural Theory of Mind Reasoners?](https://mlsanity.com/paper/2512.17394/) (2025)
- [Cognitively-Inspired Tokens Overcome Egocentric Bias in Multimodal Models](https://mlsanity.com/paper/2601.16378/) (2026)
- [Who Sees What? Structured Thought-Action Sequences for Epistemic Reasoning in LLMs](https://mlsanity.com/paper/2508.14564/) (2025)
- [Position: Artificial Intelligence Needs Meta Intelligence -- the Case for Metacognitive AI](https://mlsanity.com/paper/2605.15567/) (2026)

## Abstract (arXiv)

This study extends previous research on spatial representations in multimodal AI systems. Although current models demonstrate a rich understanding of spatial information from images, this information is rooted in propositional representations, which differ from the analog representations employed in human and animal spatial cognition. To further explore these limitations, we apply techniques from cognitive and developmental science to assess the perspective-taking abilities of GPT-4o. Our analysis enables a comparison between the cognitive development of the human brain and that of multimodal AI, offering guidance for future research and model development.