# GRATE: a Graph transformer-based deep Reinforcement learning Approach for Time-efficient autonomous robot Exploration

arXiv: https://arxiv.org/abs/2509.12863 · HTML: https://mlsanity.com/paper/2509.12863/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 23 of its 37 references that are in the map.
- First posted on arXiv 16 Sep 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [How Powerful are Graph Neural Networks?](https://mlsanity.com/paper/1810.00826/) (2018)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [Recipe for a General, Powerful, Scalable Graph Transformer](https://mlsanity.com/paper/2205.12454/) (2022)
- [Benchmarking Graph Neural Networks](https://mlsanity.com/paper/2003.00982/) (2020)
- [A Generalization of Transformer Networks to Graphs](https://mlsanity.com/paper/2012.09699/) (2020)
- [On the Bottleneck of Graph Neural Networks and its Practical Implications](https://mlsanity.com/paper/2006.05205/) (2020)
- [Ray: A Distributed Framework for Emerging AI Applications](https://mlsanity.com/paper/1712.05889/) (2017)
- [Understanding over-squashing and bottlenecks on graphs via curvature](https://mlsanity.com/paper/2111.14522/) (2021)
- [Residual Gated Graph ConvNets](https://mlsanity.com/paper/1711.07553/) (2017)
- [Representing Long-Range Context for Graph Neural Networks with Global Attention](https://mlsanity.com/paper/2201.08821/) (2022)
- [Structure-Aware Transformer for Graph Representation Learning](https://mlsanity.com/paper/2202.03036/) (2022)
- [SGFormer: Simplifying and Empowering Transformers for Large-Graph Representations](https://mlsanity.com/paper/2306.10759/) (2023)
- [Autonomous Exploration Under Uncertainty via Deep Reinforcement Learning on Graphs](https://mlsanity.com/paper/2007.12640/) (2020)
- [Autonomous Exploration Development Environment and the Planning Algorithms](https://mlsanity.com/paper/2110.14573/) (2021)
- [ARiADNE: A Reinforcement learning approach using Attention-based Deep Networks for Exploration](https://mlsanity.com/paper/2301.11575/) (2023)
- [Deep Reinforcement Learning-based Large-scale Robot Exploration](https://mlsanity.com/paper/2403.10833/) (2024)
- [HDPlanner: Advancing Autonomous Deployments in Unknown Environments through Hierarchical Decision Networks](https://mlsanity.com/paper/2408.03768/) (2024)
- [HPHS: Hierarchical Planning based on Hybrid Frontier Sampling for Unknown Environments Exploration](https://mlsanity.com/paper/2407.10660/) (2024)
- [CogniPlan: Uncertainty-Guided Path Planning with Conditional Generative Layout Prediction](https://mlsanity.com/paper/2508.03027/) (2025)
- [MapExRL: Human-Inspired Indoor Exploration with Predicted Environment Context and Reinforcement Learning](https://mlsanity.com/paper/2503.01548/) (2025)

## Abstract (arXiv)

Autonomous robot exploration (ARE) is the process of a robot autonomously navigating and mapping an unknown environment. Recent Reinforcement Learning (RL)-based approaches typically formulate ARE as a sequential decision-making problem defined on a collision-free informative graph. However, these methods often demonstrate limited reasoning ability over graph-structured data. Moreover, due to the insufficient consideration of robot motion, the resulting RL policies are generally optimized to minimize travel distance, while neglecting time efficiency. To overcome these limitations, we propose GRATE, a Deep Reinforcement Learning (DRL)-based approach that leverages a Graph Transformer to effectively capture both local structure patterns and global contextual dependencies of the informative graph, thereby enhancing the model's reasoning capability across the entire environment. In addition, we deploy a Kalman filter to smooth the waypoint outputs, ensuring that the resulting path is kinodynamically feasible for the robot to follow. Experimental results demonstrate that our method exhibits better exploration efficiency (up to 21.5% in distance and 21.3% in time to complete exploration) than state-of-the-art conventional and learning-based baselines in various simulation benchmarks. We also validate our planner in real-world scenarios.