# No RL, No Simulation: Learning to Navigate without Navigating

arXiv: https://arxiv.org/abs/2110.09470 · HTML: https://mlsanity.com/paper/2110.09470/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 48 references that are in the map.
- First posted on arXiv 18 Oct 2021.

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Conservative Q-Learning for Offline Reinforcement Learning](https://mlsanity.com/paper/2006.04779/) (2020)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments](https://mlsanity.com/paper/1711.07280/) (2017)
- [AI2-THOR: An Interactive 3D Environment for Visual AI](https://mlsanity.com/paper/1712.05474/) (2017)
- [On Evaluation of Embodied Navigation Agents](https://mlsanity.com/paper/1807.06757/) (2018)
- [Gibson Env: Real-World Perception for Embodied Agents](https://mlsanity.com/paper/1808.10654/) (2018)
- [Object Goal Navigation using Goal-Oriented Semantic Exploration](https://mlsanity.com/paper/2007.00643/) (2020)
- [Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning](https://mlsanity.com/paper/1609.05143/) (2016)
- [DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames](https://mlsanity.com/paper/1911.00357/) (2019)
- [Learning to Explore using Active Neural SLAM](https://mlsanity.com/paper/2004.05155/) (2020)
- [ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objects](https://mlsanity.com/paper/2006.13171/) (2020)
- [Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learning](https://mlsanity.com/paper/2002.08396/) (2020)
- [Cognitive Mapping and Planning for Visual Navigation](https://mlsanity.com/paper/1702.03920/) (2017)
- [Semi-parametric Topological Memory for Navigation](https://mlsanity.com/paper/1803.00653/) (2018)
- [Neural Topological SLAM for Visual Navigation](https://mlsanity.com/paper/2005.12256/) (2020)
- [Search on the Replay Buffer: Bridging Planning and Reinforcement Learning](https://mlsanity.com/paper/1906.05253/) (2019)
- [Visual Semantic Navigation using Scene Priors](https://mlsanity.com/paper/1810.06543/) (2018)
- [Learning Exploration Policies for Navigation](https://mlsanity.com/paper/1903.01959/) (2019)
- [Occupancy Anticipation for Efficient Exploration and Navigation](https://mlsanity.com/paper/2008.09285/) (2020)
- [Learning to Navigate in Cities Without a Map](https://mlsanity.com/paper/1804.00168/) (2018)
- [ViNG: Learning Open-World Navigation with Visual Goals](https://mlsanity.com/paper/2012.09812/) (2020)
- [Memory-Augmented Reinforcement Learning for Image-Goal Navigation](https://mlsanity.com/paper/2101.05181/) (2021)

## Built on by (most cited first, 30 of 65)

- [CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigation](https://mlsanity.com/paper/2203.10421/) (2022)
- [GNM: A General Navigation Model to Drive Any Robot](https://mlsanity.com/paper/2210.03370/) (2022)
- [ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2304.03047/) (2023)
- [PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learning](https://mlsanity.com/paper/2201.10029/) (2022)
- [Habitat-Matterport 3D Semantics Dataset](https://mlsanity.com/paper/2210.05633/) (2022)
- [Navigating to Objects in the Real World](https://mlsanity.com/paper/2212.00922/) (2022)
- [GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigation](https://mlsanity.com/paper/2404.06609/) (2024)
- [When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?](https://mlsanity.com/paper/2204.05618/) (2022)
- [Sim-2-Sim Transfer for Vision-and-Language Navigation in Continuous Environments](https://mlsanity.com/paper/2204.09667/) (2022)
- [Offline Visual Representation Learning for Embodied Navigation](https://mlsanity.com/paper/2204.13226/) (2022)
- [OVRL-V2: A simple state-of-art baseline for ImageNav and ObjectNav](https://mlsanity.com/paper/2303.07798/) (2023)
- [Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigation](https://mlsanity.com/paper/2202.02440/) (2022)
- [Instance-Specific Image Goal Navigation: Training Embodied Agents to Find Object Instances](https://mlsanity.com/paper/2211.15876/) (2022)
- [Renderable Neural Radiance Map for Visual Navigation](https://mlsanity.com/paper/2303.00304/) (2023)
- [Navigating to Objects Specified by Images](https://mlsanity.com/paper/2304.01192/) (2023)
- [GaussNav: Gaussian Splatting for Visual Navigation](https://mlsanity.com/paper/2403.11625/) (2024)
- [Retrospectives on the Embodied AI Workshop](https://mlsanity.com/paper/2210.06849/) (2022)
- [Learning Navigational Visual Representations with Semantic Map Supervision](https://mlsanity.com/paper/2307.12335/) (2023)
- [CityWalker: Learning Embodied Urban Navigation from Web-Scale Videos](https://mlsanity.com/paper/2411.17820/) (2024)
- [DynaMem: Online Dynamic Spatio-Semantic Memory for Open World Mobile Manipulation](https://mlsanity.com/paper/2411.04999/) (2024)
- [Prioritized Semantic Learning for Zero-shot Instance Navigation](https://mlsanity.com/paper/2403.11650/) (2024)
- [Phone2Proc: Bringing Robust Robots Into Our Chaotic World](https://mlsanity.com/paper/2212.04819/) (2022)
- [Last-Mile Embodied Visual Navigation](https://mlsanity.com/paper/2211.11746/) (2022)
- [PEANUT: Predicting and Navigating to Unseen Targets](https://mlsanity.com/paper/2212.02497/) (2022)
- [FGPrompt: Fine-grained Goal Prompting for Image-goal Navigation](https://mlsanity.com/paper/2310.07473/) (2023)
- [Know Your Boundaries: The Necessity of Explicit Behavioral Cloning in Offline RL](https://mlsanity.com/paper/2206.00695/) (2022)
- [Meta-Explore: Exploratory Hierarchical Vision-and-Language Navigation Using Scene Object Spectrum Grounding](https://mlsanity.com/paper/2303.04077/) (2023)
- [Is Imitation All You Need? Generalized Decision-Making with Dual-Phase Training](https://mlsanity.com/paper/2307.07909/) (2023)
- [NeRFs in Robotics: A Survey](https://mlsanity.com/paper/2405.01333/) (2024)
- [SAME: Learning Generic Language-Guided Visual Navigation with State-Adaptive Mixture of Experts](https://mlsanity.com/paper/2412.05552/) (2024)

## Abstract (arXiv)

Most prior methods for learning navigation policies require access to simulation environments, as they need online policy interaction and rely on ground-truth maps for rewards. However, building simulators is expensive (requires manual effort for each and every scene) and creates challenges in transferring learned policies to robotic platforms in the real-world, due to the sim-to-real domain gap. In this paper, we pose a simple question: Do we really need active interaction, ground-truth maps or even reinforcement-learning (RL) in order to solve the image-goal navigation task? We propose a self-supervised approach to learn to navigate from only passive videos of roaming. Our approach, No RL, No Simulator (NRNS), is simple and scalable, yet highly effective. NRNS outperforms RL-based formulations by a significant margin. We present NRNS as a strong baseline for any future image-based navigation tasks that use RL or Simulation.