# Explore, Map, Remember, Decide: Are Embodied VLMs Ready for Safety-Critical Scenarios?

arXiv: https://arxiv.org/abs/2608.08077 · HTML: https://mlsanity.com/paper/2608.08077/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 12 of its 30 references that are in the map.
- First posted on arXiv 8 Aug 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 12 of 12)

- [ProcTHOR: Large-Scale Embodied AI Using Procedural Generation](https://mlsanity.com/paper/2206.06994/) (2022)
- [What's "up" with vision-language models? Investigating their struggle with spatial reasoning](https://mlsanity.com/paper/2310.19785/) (2023)
- [BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments](https://mlsanity.com/paper/2108.03332/) (2021)
- [Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making](https://mlsanity.com/paper/2410.07166/) (2024)
- [Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawing](https://mlsanity.com/paper/2506.09965/) (2025)
- [Visual Room Rearrangement](https://mlsanity.com/paper/2103.16544/) (2021)
- [Spatial Reasoning with Vision-Language Models in Ego-Centric Multi-View Scenes](https://mlsanity.com/paper/2509.06266/) (2025)
- [VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thought](https://mlsanity.com/paper/2406.14596/) (2024)
- [Emergence of Maps in the Memories of Blind Navigation Agents](https://mlsanity.com/paper/2301.13261/) (2023)
- [Theory of Space: Can Foundation Models Construct Spatial Beliefs through Active Exploration?](https://mlsanity.com/paper/2602.07055/) (2026)
- [DivScene: Towards Open-Vocabulary Object Navigation with Large Vision Language Models in Diverse Scenes](https://mlsanity.com/paper/2410.02730/) (2024)
- [eMEM: A Hybrid Spatio-Temporal Memory System For Embodied Agents](https://mlsanity.com/paper/2606.03374/) (2026)

## Abstract (arXiv)

Theory of Space framework (ToS) assesses the spatial understanding of curiosity-driven Vision-Language Models (VLMs) under partial observability. As AI techniques are increasingly applied to safety-critical scenarios, it is crucial to understand whether VLMs possess robust spatial memory and make reliable decisions. In this paper, we assess whether VLMs' decisions are based on physical evidence or are corrupted by visual-language biases, if their memory processes align with human cognitive patterns, and how they respond to environmental hazards. We extend the ToS framework into a safety-critical, goal-driven pipeline, named Explore, Map, Remember, and Decide (EMRD). We then quantify Exploration Competence (Explore) through metrics of environmental coverage and temporal efficiency, assess Spatial Fidelity (Map), evaluate, with a suite of psychological metrics, Memory Persistence (Remember), and measure, using focal-point metrics, Cognitive Decision-Making (Decide). Our results show that in terms of decision-making capabilities, VLMs frequently select evacuation points based on pre-trained textual priors while lacking the spatial grounding to justify their choices. We also show that spatial reasoning degrades in low-light conditions, but it is not affected by texture and colour tampering. Our findings suggest that VLM memory fundamentally diverges from human cognition, creating unpredictable risks of misalignment.