# Learning Coverage Paths in Unknown Environments with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2306.16978 · HTML: https://mlsanity.com/paper/2306.16978/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 42 references that are in the map.
- First posted on arXiv 29 Jun 2023.

## Builds on (most cited first, 11 of 11)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey](https://mlsanity.com/paper/2003.04960/) (2020)
- [Learning to Explore using Active Neural SLAM](https://mlsanity.com/paper/2004.05155/) (2020)
- [Learning Exploration Policies for Navigation](https://mlsanity.com/paper/1903.01959/) (2019)
- [Virtual-to-real Deep Reinforcement Learning: Continuous Control of Mobile Robots for Mapless Navigation](https://mlsanity.com/paper/1703.00420/) (2017)
- [Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generation](https://mlsanity.com/paper/1806.10729/) (2018)
- [UAV Coverage Path Planning under Varying Power Constraints using Deep Reinforcement Learning](https://mlsanity.com/paper/2003.02609/) (2020)
- [Explore-Bench: Data Sets, Metrics and Evaluations for Frontier-based and Deep-reinforcement-learning-based Autonomous Exploration](https://mlsanity.com/paper/2202.11931/) (2022)
- [Reinforcement Learning-Based Coverage Path Planning with Implicit Cellular Decomposition](https://mlsanity.com/paper/2110.09018/) (2021)
- [Planning Hybrid Driving-Stepping Locomotion on Multiple Levels of Abstraction](https://mlsanity.com/paper/1809.07058/) (2018)

## Built on by (most cited first, 5 of 5)

- [Journey into Automation: Image-Derived Pavement Texture Extraction and Evaluation](https://mlsanity.com/paper/2501.02414/) (2025)
- [MaskPlanner: Learning-Based Object-Centric Motion Generation from 3D Point Clouds](https://mlsanity.com/paper/2502.18745/) (2025)
- [Sim-to-Real Transfer of Deep Reinforcement Learning Agents for Online Coverage Path Planning](https://mlsanity.com/paper/2406.04920/) (2024)
- [Baseline-Free Policy Optimization for Neural Combinatorial Optimization](https://mlsanity.com/paper/2606.10321/) (2026)
- [IMAGINE: Intelligent Multi-Agent Godot-based Indoor Networked Exploration](https://mlsanity.com/paper/2602.02858/) (2026)

## Abstract (arXiv)

Coverage path planning (CPP) is the problem of finding a path that covers the entire free space of a confined area, with applications ranging from robotic lawn mowing to search-and-rescue. When the environment is unknown, the path needs to be planned online while mapping the environment, which cannot be addressed by offline planning methods that do not allow for a flexible path space. We investigate how suitable reinforcement learning is for this challenging problem, and analyze the involved components required to efficiently learn coverage paths, such as action space, input feature representation, neural network architecture, and reward function. We propose a computationally feasible egocentric map representation based on frontiers, and a novel reward term based on total variation to promote complete coverage. Through extensive experiments, we show that our approach surpasses the performance of both previous RL-based approaches and highly specialized methods across multiple CPP variations.