# Deep Reinforcement Learning-based Large-scale Robot Exploration

arXiv: https://arxiv.org/abs/2403.10833 · HTML: https://mlsanity.com/paper/2403.10833/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 17 of its 36 references that are in the map.
- First posted on arXiv 16 Mar 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [Ray: A Distributed Framework for Emerging AI Applications](https://mlsanity.com/paper/1712.05889/) (2017)
- [Soft Actor-Critic for Discrete Action Settings](https://mlsanity.com/paper/1910.07207/) (2019)
- [FUEL: Fast UAV Exploration using Incremental Frontier Structure and Hierarchical Planning](https://mlsanity.com/paper/2010.11561/) (2020)
- [An Efficient Sampling-based Method for Online Informative Path Planning in Unknown Environments](https://mlsanity.com/paper/1909.09548/) (2019)
- [Deep Reinforcement Learning based Automatic Exploration for Navigation in Unknown Environment](https://mlsanity.com/paper/2007.11808/) (2020)
- [Autonomous Exploration Under Uncertainty via Deep Reinforcement Learning on Graphs](https://mlsanity.com/paper/2007.12640/) (2020)
- [ARiADNE: A Reinforcement learning approach using Attention-based Deep Networks for Exploration](https://mlsanity.com/paper/2301.11575/) (2023)
- [Semantic OcTree Mapping and Shannon Mutual Information Computation for Robot Exploration](https://mlsanity.com/paper/2112.04063/) (2021)
- [FSMI: Fast computation of Shannon Mutual Information for information-theoretic mapping](https://mlsanity.com/paper/1905.02238/) (2019)
- [Explore-Bench: Data Sets, Metrics and Evaluations for Frontier-based and Deep-reinforcement-learning-based Autonomous Exploration](https://mlsanity.com/paper/2202.11931/) (2022)
- [DAN: Decentralized Attention-based Neural Network for the MinMax Multiple Traveling Salesman Problem](https://mlsanity.com/paper/2109.04205/) (2021)
- [Fast and Compute-efficient Sampling-based Local Exploration Planning via Distribution Learning](https://mlsanity.com/paper/2202.13715/) (2022)
- [Graph-Based Topological Exploration Planning in Large-Scale 3D Environments](https://mlsanity.com/paper/2103.16829/) (2021)
- [Active Mapping via Gradient Ascent Optimization of Shannon Mutual Information over Continuous SE(3) Trajectories](https://mlsanity.com/paper/2204.07623/) (2022)

## Built on by (most cited first, 16 of 16)

- [DARE: Diffusion Policy for Autonomous Robot Exploration](https://mlsanity.com/paper/2410.16687/) (2024)
- [GLEAM: Learning Generalizable Exploration Policy for Active Mapping in Complex 3D Indoor Scenes](https://mlsanity.com/paper/2505.20294/) (2025)
- [Can a Robot Walk the Robotic Dog: Triple-Zero Collaborative Navigation for Heterogeneous Multi-Agent Systems](https://mlsanity.com/paper/2603.21723/) (2026)
- [Towards Robust Multi-UAV Collaboration: MARL with Noise-Resilient Communication and Attention Mechanisms](https://mlsanity.com/paper/2503.02913/) (2025)
- [DiCriTest: Testing Scenario Generation for Decision-Making Agents Considering Diversity and Criticality](https://mlsanity.com/paper/2508.11514/) (2025)
- [HEADER: Hierarchical Robot Exploration via Attention-Based Deep Reinforcement Learning with Expert-Guided Reward](https://mlsanity.com/paper/2510.15679/) (2025)
- [FARE: Fast-Slow Agentic Robotic Exploration](https://mlsanity.com/paper/2601.14681/) (2026)
- [Privileged Reinforcement and Communication Learning for Distributed, Bandwidth-limited Multi-robot Exploration](https://mlsanity.com/paper/2407.20203/) (2024)
- [Cooperative and Asynchronous Transformer-based Mission Planning for Heterogeneous Teams of Mobile Robots](https://mlsanity.com/paper/2410.06372/) (2024)
- [DyPNIPP: Predicting Environment Dynamics for RL-based Robust Informative Path Planning](https://mlsanity.com/paper/2410.17186/) (2024)
- [GRATE: a Graph transformer-based deep Reinforcement learning Approach for Time-efficient autonomous robot Exploration](https://mlsanity.com/paper/2509.12863/) (2025)
- [SuperEx: Enhancing Indoor Mapping and Exploration using Non-Line-of-Sight Perception](https://mlsanity.com/paper/2510.10506/) (2025)
- [Dual-Agent Reinforcement Learning for Adaptive and Cost-Aware Visual-Inertial Odometry](https://mlsanity.com/paper/2511.21083/) (2025)
- [MA3DSG: Multi-Agent 3D Scene Graph Generation for Large-Scale Indoor Environments](https://mlsanity.com/paper/2602.04152/) (2026)
- [E2DT: Efficient and Effective Decision Transformer with Experience-Aware Sampling for Robotic Manipulation](https://mlsanity.com/paper/2605.00159/) (2026)
- [VANDERER: Map-Free Exploration using Future-Aware and Visual-Curiosity-Guided Diffusion Policy](https://mlsanity.com/paper/2606.14879/) (2026)

## Abstract (arXiv)

In this work, we propose a deep reinforcement learning (DRL) based reactive planner to solve large-scale Lidar-based autonomous robot exploration problems in 2D action space. Our DRL-based planner allows the agent to reactively plan its exploration path by making implicit predictions about unknown areas, based on a learned estimation of the underlying transition model of the environment. To this end, our approach relies on learned attention mechanisms for their powerful ability to capture long-term dependencies at different spatial scales to reason about the robot's entire belief over known areas. Our approach relies on ground truth information (i.e., privileged learning) to guide the environment estimation during training, as well as on a graph rarefaction algorithm, which allows models trained in small-scale environments to scale to large-scale ones. Simulation results show that our model exhibits better exploration efficiency (12% in path length, 6% in makespan) and lower planning time (60%) than the state-of-the-art planners in a 130m x 100m benchmark scenario. We also validate our learned model on hardware.