# mindmap: Spatial Memory in Deep Feature Maps for 3D Action Policies

arXiv: https://arxiv.org/abs/2509.20297 · HTML: https://mlsanity.com/paper/2509.20297/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Sep 2025.

## Built on by (most cited first, 5 of 5)

- [AERMANI-VLM: Structured Prompting and Reasoning for Aerial Manipulation with Vision Language Models](https://mlsanity.com/paper/2511.01472/) (2025)
- [Understanding the Impact of Geometric Foundation Models on Vision-Language-Action Models](https://mlsanity.com/paper/2605.24642/) (2026)
- [SERF: Spatiotemporal Environment and Robot Feature Map for Long-Horizon Mobile Manipulation](https://mlsanity.com/paper/2606.12956/) (2026)
- [WeaveRL: Weaving Reconstruction into Scene-Aware Fabrics for Perceptive Reinforcement Learning](https://mlsanity.com/paper/2609.18685/) (2026)
- [Self-Supervised Keyframe Discovery for Horizon-Invariant Behavior Cloning](https://mlsanity.com/paper/2610.10857/) (2026)

## Abstract (arXiv)

End-to-end learning of robot control policies, structured as neural networks, has emerged as a promising approach to robotic manipulation. To complete many common tasks, relevant objects are required to pass in and out of a robot's field of view. In these settings, spatial memory - the ability to remember the spatial composition of the scene - is an important competency. However, building such mechanisms into robot learning systems remains an open research problem. We introduce mindmap (Spatial Memory in Deep Feature Maps for 3D Action Policies), a 3D diffusion policy that generates robot trajectories based on a semantic 3D reconstruction of the environment. We show in simulation experiments that our approach is effective at solving tasks where state-of-the-art approaches without memory mechanisms struggle. We release our reconstruction system, training code, and evaluation tasks to spur research in this direction.