# Teaching Agents how to Map: Spatial Reasoning for Multi-Object Navigation

arXiv: https://arxiv.org/abs/2107.06011 · HTML: https://mlsanity.com/paper/2107.06011/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 52 references that are in the map.
- First posted on arXiv 13 Jul 2021.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [Neural Turing Machines](https://mlsanity.com/paper/1410.5401/) (2014)
- [On Evaluation of Embodied Navigation Agents](https://mlsanity.com/paper/1807.06757/) (2018)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [Reinforcement Learning with Unsupervised Auxiliary Tasks](https://mlsanity.com/paper/1611.05397/) (2016)
- [Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning](https://mlsanity.com/paper/1609.05143/) (2016)
- [DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames](https://mlsanity.com/paper/1911.00357/) (2019)
- [Learning to Explore using Active Neural SLAM](https://mlsanity.com/paper/2004.05155/) (2020)
- [Cognitive Mapping and Planning for Visual Navigation](https://mlsanity.com/paper/1702.03920/) (2017)
- [Semi-parametric Topological Memory for Navigation](https://mlsanity.com/paper/1803.00653/) (2018)
- [Learning to Navigate in Complex Environments](https://mlsanity.com/paper/1611.03673/) (2016)
- [Neural Topological SLAM for Visual Navigation](https://mlsanity.com/paper/2005.12256/) (2020)
- [ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learning](https://mlsanity.com/paper/1605.02097/) (2016)
- [Goal-conditioned Imitation Learning](https://mlsanity.com/paper/1906.05838/) (2019)
- [Scene Memory Transformer for Embodied Agents in Long-Horizon Tasks](https://mlsanity.com/paper/1903.03878/) (2019)
- [Playing FPS Games with Deep Reinforcement Learning](https://mlsanity.com/paper/1609.05521/) (2016)
- [MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigation](https://mlsanity.com/paper/2012.03912/) (2020)
- [Neural Map: Structured Memory for Deep Reinforcement Learning](https://mlsanity.com/paper/1702.08360/) (2017)
- [DeepMind Lab](https://mlsanity.com/paper/1612.03801/) (2016)
- [Control of Memory, Active Perception, and Action in Minecraft](https://mlsanity.com/paper/1605.09128/) (2016)
- [Auxiliary Tasks Speed Up Learning PointGoal Navigation](https://mlsanity.com/paper/2007.04561/) (2020)
- [Learning to Act by Predicting the Future](https://mlsanity.com/paper/1611.01779/) (2016)
- [Learning to plan with uncertain topological maps](https://mlsanity.com/paper/2007.05270/) (2020)
- [Benchmarking Classic and Learned Navigation in Complex 3D Environments](https://mlsanity.com/paper/1901.10915/) (2019)
- [EgoMap: Projective mapping and structured egocentric memory for Deep RL](https://mlsanity.com/paper/2002.02286/) (2020)

## Built on by (most cited first, 17 of 17)

- [Retrospectives on the Embodied AI Workshop](https://mlsanity.com/paper/2210.06849/) (2022)
- [Multi-Object Navigation with dynamically learned neural implicit representations](https://mlsanity.com/paper/2210.05129/) (2022)
- [Learning Active Camera for Multi-Object Navigation](https://mlsanity.com/paper/2210.07505/) (2022)
- [End-to-End (Instance)-Image Goal Navigation through Correspondence as an Emergent Phenomenon](https://mlsanity.com/paper/2309.16634/) (2023)
- [One Map to Find Them All: Real-time Open-Vocabulary Mapping for Zero-shot Multi-Object Navigation](https://mlsanity.com/paper/2409.11764/) (2024)
- [MOPA: Modular Object Navigation with PointGoal Agents](https://mlsanity.com/paper/2304.03696/) (2023)
- [AutoNeRF: Training Implicit Scene Representations with Autonomous Agents](https://mlsanity.com/paper/2304.11241/) (2023)
- [Object Manipulation via Visual Target Localization](https://mlsanity.com/paper/2203.08141/) (2022)
- [Learning with a Mole: Transferable latent spatial representations for navigation without reconstruction](https://mlsanity.com/paper/2306.03857/) (2023)
- [Sequence-Agnostic Multi-Object Navigation](https://mlsanity.com/paper/2305.06178/) (2023)
- [Find Everything: A General Vision Language Model Approach to Multi-Object Search](https://mlsanity.com/paper/2410.00388/) (2024)
- [Multi-Object Navigation in real environments using hybrid policies](https://mlsanity.com/paper/2401.13800/) (2024)
- [Language-Based Augmentation to Address Shortcut Learning in Object Goal Navigation](https://mlsanity.com/paper/2402.05090/) (2024)
- [RANa: Retrieval-Augmented Navigation](https://mlsanity.com/paper/2504.03524/) (2025)
- [Kinaema: a recurrent sequence model for memory and pose in motion](https://mlsanity.com/paper/2510.20261/) (2025)
- [Long-HOT: A Modular Hierarchical Approach for Long-Horizon Object Transport](https://mlsanity.com/paper/2210.15908/) (2022)
- [Zero-BEV: Zero-shot Projection of Any First-Person Modality to BEV Maps](https://mlsanity.com/paper/2402.13848/) (2024)

## Abstract (arXiv)

In the context of visual navigation, the capacity to map a novel environment is necessary for an agent to exploit its observation history in the considered place and efficiently reach known goals. This ability can be associated with spatial reasoning, where an agent is able to perceive spatial relationships and regularities, and discover object characteristics. Recent work introduces learnable policies parametrized by deep neural networks and trained with Reinforcement Learning (RL). In classical RL setups, the capacity to map and reason spatially is learned end-to-end, from reward alone. In this setting, we introduce supplementary supervision in the form of auxiliary tasks designed to favor the emergence of spatial perception capabilities in agents trained for a goal-reaching downstream objective. We show that learning to estimate metrics quantifying the spatial relationships between an agent at a given location and a goal to reach has a high positive impact in Multi-Object Navigation settings. Our method significantly improves the performance of different baseline agents, that either build an explicit or implicit representation of the environment, even matching the performance of incomparable oracle agents taking ground-truth maps as input. A learning-based agent from the literature trained with the proposed auxiliary losses was the winning entry to the Multi-Object Navigation Challenge, part of the CVPR 2021 Embodied AI Workshop.