# Learning Exploration Policies for Navigation

arXiv: https://arxiv.org/abs/1903.01959 · HTML: https://mlsanity.com/paper/1903.01959/

## Key facts

- Cited by 116 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Mar 2019.

## Built on by (most cited first, 30 of 116)

- [Navigation World Models](https://mlsanity.com/paper/2412.03572/) (2024)
- [A Survey of Embodied AI: From Simulators to Research Tasks](https://mlsanity.com/paper/2103.04918/) (2021)
- [NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration](https://mlsanity.com/paper/2310.07896/) (2023)
- [CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigation](https://mlsanity.com/paper/2203.10421/) (2022)
- [Open-vocabulary Queryable Scene Representations for Real World Planning](https://mlsanity.com/paper/2209.09874/) (2022)
- [PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learning](https://mlsanity.com/paper/2201.10029/) (2022)
- [Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigation](https://mlsanity.com/paper/2203.02764/) (2022)
- [Habitat-Web: Learning Embodied Object-Search Strategies from Human Demonstrations at Scale](https://mlsanity.com/paper/2204.03514/) (2022)
- [Visual Room Rearrangement](https://mlsanity.com/paper/2103.16544/) (2021)
- [Bird's-Eye-View Scene Graph for Vision-Language Navigation](https://mlsanity.com/paper/2308.04758/) (2023)
- [Memory-Augmented Reinforcement Learning for Image-Goal Navigation](https://mlsanity.com/paper/2101.05181/) (2021)
- [The ThreeDWorld Transport Challenge: A Visually Guided Task-and-Motion Planning Benchmark for Physically Realistic Embodied AI](https://mlsanity.com/paper/2103.14025/) (2021)
- [No RL, No Simulation: Learning to Navigate without Navigating](https://mlsanity.com/paper/2110.09470/) (2021)
- [See and Think: Embodied Agent in Virtual Environment](https://mlsanity.com/paper/2311.15209/) (2023)
- [Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networks](https://mlsanity.com/paper/2108.01176/) (2021)
- [Renderable Neural Radiance Map for Visual Navigation](https://mlsanity.com/paper/2303.00304/) (2023)
- [Learning to Map for Active Semantic Goal Navigation](https://mlsanity.com/paper/2106.15648/) (2021)
- [NeRF2Real: Sim2real Transfer of Vision-guided Bipedal Motion Skills using Neural Radiance Fields](https://mlsanity.com/paper/2210.04932/) (2022)
- [GenNBV: Generalizable Next-Best-View Policy for Active 3D Reconstruction](https://mlsanity.com/paper/2402.16174/) (2024)
- [Hierarchical Object-to-Zone Graph for Object Navigation](https://mlsanity.com/paper/2109.02066/) (2021)
- [Uncertainty-driven Planner for Exploration and Navigation](https://mlsanity.com/paper/2202.11907/) (2022)
- [Stubborn: A Strong Baseline for Indoor Object Navigation](https://mlsanity.com/paper/2203.07359/) (2022)
- [TIDEE: Tidying Up Novel Rooms using Visuo-Semantic Commonsense Priors](https://mlsanity.com/paper/2207.10761/) (2022)
- [SPIN: Simultaneous Perception, Interaction and Navigation](https://mlsanity.com/paper/2405.07991/) (2024)
- [Last-Mile Embodied Visual Navigation](https://mlsanity.com/paper/2211.11746/) (2022)
- [RoboHop: Segment-based Topological Map Representation for Open-World Visual Navigation](https://mlsanity.com/paper/2405.05792/) (2024)
- [Hierarchical Auto-Organizing System for Open-Ended Multi-Agent Navigation](https://mlsanity.com/paper/2403.08282/) (2024)
- [Causal Navigation by Continuous-time Neural Networks](https://mlsanity.com/paper/2106.08314/) (2021)
- [Successor Feature Landmarks for Long-Horizon Goal-Conditioned Reinforcement Learning](https://mlsanity.com/paper/2111.09858/) (2021)
- [EgoEnv: Human-centric environment representations from egocentric video](https://mlsanity.com/paper/2207.11365/) (2022)

## Abstract (arXiv)

Numerous past works have tackled the problem of task-driven navigation. But, how to effectively explore a new environment to enable a variety of down-stream tasks has received much less attention. In this work, we study how agents can autonomously explore realistic and complex 3D environments without the context of task-rewards. We propose a learning-based approach and investigate different policy architectures, reward functions, and training paradigms. We find that the use of policies with spatial memory that are bootstrapped with imitation learning and finally finetuned with coverage rewards derived purely from on-board sensors can be effective at exploring novel environments. We show that our learned exploration policies can explore better than classical approaches based on geometry alone and generic learning-based exploration techniques. Finally, we also show how such task-agnostic exploration can be used for down-stream tasks. Code and Videos are available at: https://sites.google.com/view/exploration-for-nav.