# Learning to Navigate in Complex Environments

arXiv: https://arxiv.org/abs/1611.03673 · HTML: https://mlsanity.com/paper/1611.03673/

## Key facts

- Cited by 198 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2016.

## Built on by (most cited first, 30 of 198)

- [Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation](https://mlsanity.com/paper/1811.10092/) (2018)
- [Navigation World Models](https://mlsanity.com/paper/2412.03572/) (2024)
- [Masked Visual Pre-training for Motor Control](https://mlsanity.com/paper/2203.06173/) (2022)
- [Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer](https://mlsanity.com/paper/2207.14024/) (2022)
- [Agent AI: Surveying the Horizons of Multimodal Interaction](https://mlsanity.com/paper/2401.03568/) (2024)
- [Exploration in Deep Reinforcement Learning: A Survey](https://mlsanity.com/paper/2205.00824/) (2022)
- [Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functions](https://mlsanity.com/paper/2309.10150/) (2023)
- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPs](https://mlsanity.com/paper/2110.05038/) (2021)
- [Navigating to Objects in the Real World](https://mlsanity.com/paper/2212.00922/) (2022)
- [Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentation](https://mlsanity.com/paper/2107.00644/) (2021)
- [ManipulaTHOR: A Framework for Visual Object Manipulation](https://mlsanity.com/paper/2104.11213/) (2021)
- [Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers](https://mlsanity.com/paper/2107.03996/) (2021)
- [Auxiliary Tasks and Exploration Enable ObjectNav](https://mlsanity.com/paper/2104.04112/) (2021)
- [DriveCoT: Integrating Chain-of-Thought Reasoning with End-to-End Driving](https://mlsanity.com/paper/2403.16996/) (2024)
- [Memory-Augmented Reinforcement Learning for Image-Goal Navigation](https://mlsanity.com/paper/2101.05181/) (2021)
- [The ThreeDWorld Transport Challenge: A Visually Guided Task-and-Motion Planning Benchmark for Physically Realistic Embodied AI](https://mlsanity.com/paper/2103.14025/) (2021)
- [On The Effect of Auxiliary Tasks on Representation Dynamics](https://mlsanity.com/paper/2102.13089/) (2021)
- [Deep Federated Learning for Autonomous Driving](https://mlsanity.com/paper/2110.05754/) (2021)
- [Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networks](https://mlsanity.com/paper/2108.01176/) (2021)
- [VTNet: Visual Transformer Network for Object Goal Navigation](https://mlsanity.com/paper/2105.09447/) (2021)
- [Scene-Intuitive Agent for Remote Embodied Visual Grounding](https://mlsanity.com/paper/2103.12944/) (2021)
- [Retrospectives on the Embodied AI Workshop](https://mlsanity.com/paper/2210.06849/) (2022)
- [Learning Navigational Visual Representations with Semantic Map Supervision](https://mlsanity.com/paper/2307.12335/) (2023)
- [Augmenting Automated Game Testing with Deep Reinforcement Learning](https://mlsanity.com/paper/2103.15819/) (2021)
- [Return-Based Contrastive Representation Learning for Reinforcement Learning](https://mlsanity.com/paper/2102.10960/) (2021)
- [Discovering Diverse Multi-Agent Strategic Behavior via Reward Randomization](https://mlsanity.com/paper/2103.04564/) (2021)
- [360-GS: Layout-guided Panoramic Gaussian Splatting For Indoor Roaming](https://mlsanity.com/paper/2402.00763/) (2024)
- [Hierarchical Object-to-Zone Graph for Object Navigation](https://mlsanity.com/paper/2109.02066/) (2021)
- [Chain of Thought Imitation with Procedure Cloning](https://mlsanity.com/paper/2205.10816/) (2022)

## Abstract (arXiv)

Learning to navigate in complex environments with dynamic elements is an important milestone in developing AI agents. In this work we formulate the navigation question as a reinforcement learning problem and show that data efficiency and task performance can be dramatically improved by relying on additional auxiliary tasks leveraging multimodal sensory inputs. In particular we consider jointly learning the goal-driven reinforcement learning problem with auxiliary depth prediction and loop closure classification tasks. This approach can learn to navigate from raw sensory input in complicated 3D mazes, approaching human-level performance even under conditions where the goal location changes frequently. We provide detailed analysis of the agent behaviour, its ability to localise, and its network activity dynamics, showing that the agent implicitly learns key navigation abilities.