# Zero-Shot Reinforcement Learning on Graphs for Autonomous Exploration Under Uncertainty

arXiv: https://arxiv.org/abs/2105.04758 · HTML: https://mlsanity.com/paper/2105.04758/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 34 references that are in the map.
- First posted on arXiv 11 May 2021.

## Builds on (most cited first, 13 of 13)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [Graph U-Nets](https://mlsanity.com/paper/1905.05178/) (2019)
- [StarCraft II: A New Challenge for Reinforcement Learning](https://mlsanity.com/paper/1708.04782/) (2017)
- [Graph networks as learnable physics engines for inference and control](https://mlsanity.com/paper/1806.01242/) (2018)
- [SegMatch: Segment based loop-closure for 3D point clouds](https://mlsanity.com/paper/1609.07720/) (2016)
- [A Behavioral Approach to Visual Navigation with Graph Localization Networks](https://mlsanity.com/paper/1903.00445/) (2019)
- [Autonomous Exploration Under Uncertainty via Deep Reinforcement Learning on Graphs](https://mlsanity.com/paper/2007.12640/) (2020)
- [Gaussian Process Autonomous Mapping and Exploration for Range Sensing Mobile Robots](https://mlsanity.com/paper/1605.00335/) (2016)
- [One-Shot Reinforcement Learning for Robot Navigation with Interactive Replay](https://mlsanity.com/paper/1711.10137/) (2017)
- [Zero-Shot Reinforcement Learning with Deep Attention Convolutional Neural Networks](https://mlsanity.com/paper/2001.00605/) (2020)

## Built on by (most cited first, 7 of 7)

- [Semantically-driven Deep Reinforcement Learning for Inspection Path Planning](https://mlsanity.com/paper/2505.14443/) (2025)
- [Graph learning in robotics: a survey](https://mlsanity.com/paper/2310.04294/) (2023)
- [SEAL: Simultaneous Exploration and Localization in Multi-Robot Systems](https://mlsanity.com/paper/2306.12623/) (2023)
- [Intrinsically motivated graph exploration using network theories of human curiosity](https://mlsanity.com/paper/2307.04962/) (2023)
- [Improving Reliable Navigation under Uncertainty via Predictions Informed by Non-Local Information](https://mlsanity.com/paper/2307.14501/) (2023)
- [Evaluating Generalization Mechanisms in Autonomous Cyber Attack Agents](https://mlsanity.com/paper/2603.10041/) (2026)
- [Learning-Guided Sparsification of Dynamic Graphs in Robotic Exploration](https://mlsanity.com/paper/2604.16509/) (2026)

## Abstract (arXiv)

This paper studies the problem of autonomous exploration under localization uncertainty for a mobile robot with 3D range sensing. We present a framework for self-learning a high-performance exploration policy in a single simulation environment, and transferring it to other environments, which may be physical or virtual. Recent work in transfer learning achieves encouraging performance by domain adaptation and domain randomization to expose an agent to scenarios that fill the inherent gaps in sim2sim and sim2real approaches. However, it is inefficient to train an agent in environments with randomized conditions to learn the important features of its current state. An agent can use domain knowledge provided by human experts to learn efficiently. We propose a novel approach that uses graph neural networks in conjunction with deep reinforcement learning, enabling decision-making over graphs containing relevant exploration information provided by human experts to predict a robot's optimal sensing action in belief space. The policy, which is trained only in a single simulation environment, offers a real-time, scalable, and transferable decision-making strategy, resulting in zero-shot transfer to other simulation environments and even real-world environments.