# Autonomous Exploration Under Uncertainty via Deep Reinforcement Learning on Graphs

arXiv: https://arxiv.org/abs/2007.12640 · HTML: https://mlsanity.com/paper/2007.12640/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jul 2020.

## Built on by (most cited first, 30 of 30)

- [Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networks](https://mlsanity.com/paper/2108.01176/) (2021)
- [RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulation](https://mlsanity.com/paper/2402.15487/) (2024)
- [Deep Reinforcement Learning-based Large-scale Robot Exploration](https://mlsanity.com/paper/2403.10833/) (2024)
- [Adaptive Informative Path Planning Using Deep Reinforcement Learning for UAV-based Active Sensing](https://mlsanity.com/paper/2109.13570/) (2021)
- [Graph Neural Networks for Multi-Robot Active Information Acquisition](https://mlsanity.com/paper/2209.12091/) (2022)
- [AcTExplore: Active Tactile Exploration of Unknown Objects](https://mlsanity.com/paper/2310.08745/) (2023)
- [DARE: Diffusion Policy for Autonomous Robot Exploration](https://mlsanity.com/paper/2410.16687/) (2024)
- [An Informative Path Planning Framework for Active Learning in UAV-based Semantic Mapping](https://mlsanity.com/paper/2302.03347/) (2023)
- [Zero-Shot Reinforcement Learning on Graphs for Autonomous Exploration Under Uncertainty](https://mlsanity.com/paper/2105.04758/) (2021)
- [Off-Policy Evaluation with Online Adaptation for Robot Exploration in Challenging Environments](https://mlsanity.com/paper/2204.03140/) (2022)
- [Semi-Supervised Active Learning for Semantic Segmentation in Unknown Environments Using Informative Path Planning](https://mlsanity.com/paper/2312.04402/) (2023)
- [Graph learning in robotics: a survey](https://mlsanity.com/paper/2310.04294/) (2023)
- [Uncertainty-driven 3D Gaussian Splatting Active Mapping via Anisotropic Visibility Field](https://mlsanity.com/paper/2605.30342/) (2026)
- [Learning Continuous Control Policies for Information-Theoretic Active Perception](https://mlsanity.com/paper/2209.12427/) (2022)
- [Policy Learning for Active Target Tracking over Continuous SE(3) Trajectories](https://mlsanity.com/paper/2212.01498/) (2022)
- [SEAL: Simultaneous Exploration and Localization in Multi-Robot Systems](https://mlsanity.com/paper/2306.12623/) (2023)
- [Graph Convolutional Memory using Topological Priors](https://mlsanity.com/paper/2106.14117/) (2021)
- [MapExRL: Human-Inspired Indoor Exploration with Predicted Environment Context and Reinforcement Learning](https://mlsanity.com/paper/2503.01548/) (2025)
- [Towards Robust Multi-UAV Collaboration: MARL with Noise-Resilient Communication and Attention Mechanisms](https://mlsanity.com/paper/2503.02913/) (2025)
- [HEADER: Hierarchical Robot Exploration via Attention-Based Deep Reinforcement Learning with Expert-Guided Reward](https://mlsanity.com/paper/2510.15679/) (2025)
- [FARE: Fast-Slow Agentic Robotic Exploration](https://mlsanity.com/paper/2601.14681/) (2026)
- [Improving Reliable Navigation under Uncertainty via Predictions Informed by Non-Local Information](https://mlsanity.com/paper/2307.14501/) (2023)
- [CARE: Confidence-rich Autonomous Robot Exploration using Bayesian Kernel Inference and Optimization](https://mlsanity.com/paper/2309.05200/) (2023)
- [Trajectory Planning of Robotic Manipulator in Dynamic Environment Exploiting DRL](https://mlsanity.com/paper/2403.16652/) (2024)
- [Attention-based Learning for 3D Informative Path Planning](https://mlsanity.com/paper/2506.08434/) (2025)
- [GRATE: a Graph transformer-based deep Reinforcement learning Approach for Time-efficient autonomous robot Exploration](https://mlsanity.com/paper/2509.12863/) (2025)
- [DRL-TH: Jointly Utilizing Temporal Graph Attention and Hierarchical Fusion for UGV Navigation in Crowded Environments](https://mlsanity.com/paper/2512.24284/) (2025)
- [Massive Parallel Deep Reinforcement Learning for Active SLAM](https://mlsanity.com/paper/2603.25834/) (2026)
- [Learning-Guided Sparsification of Dynamic Graphs in Robotic Exploration](https://mlsanity.com/paper/2604.16509/) (2026)
- [Scout-Assisted Planning for Heterogeneous Robot Teams under Partially Known Environments](https://mlsanity.com/paper/2605.22693/) (2026)

## Abstract (arXiv)

We consider an autonomous exploration problem in which a range-sensing mobile robot is tasked with accurately mapping the landmarks in an a priori unknown environment efficiently in real-time; it must choose sensing actions that both curb localization uncertainty and achieve information gain. For this problem, belief space planning methods that forward-simulate robot sensing and estimation may often fail in real-time implementation, scaling poorly with increasing size of the state, belief and action spaces. We propose a novel approach that uses graph neural networks (GNNs) in conjunction with deep reinforcement learning (DRL), enabling decision-making over graphs containing exploration information to predict a robot's optimal sensing action in belief space. The policy, which is trained in different random environments without human intervention, offers a real-time, scalable decision-making process whose high-performance exploratory sensing actions yield accurate maps and high rates of information gain.