mlsanityData to 14 Oct 2026

Sim-to-Real Transfer of Deep Reinforcement Learning Agents for Online Coverage Path Planning

arXiv 2406.04920 ↗First posted 7 Jun 2024cs.RO cs.LG cs.SYcited by 2

On the map: Deep reinforcement learning methods → RL for robot navigation and UAV control

Abstract (arXiv)

Coverage path planning (CPP) is the problem of finding a path that covers the entire free space of a confined area, with applications ranging from robotic lawn mowing to search-and-rescue. While for known environments, offline methods can find provably complete paths, and in some cases optimal solutions, unknown environments need to be planned online during mapping. We investigate the suitability of continuous-space reinforcement learning (RL) for this challenging problem, and propose a computationally feasible egocentric map representation based on frontiers, as well as a novel reward term based on total variation to promote complete coverage. Compared to existing classical methods, this approach allows for a flexible path space, and enables the agent to adapt to specific environment characteristics. Meanwhile, the deployment of RL models on real robot systems is difficult. Training from scratch may be infeasible due to slow convergence times, while transferring from simulation to reality, i.e. sim-to-real transfer, is a key challenge in itself. We bridge the sim-to-real gap through a semi-virtual environment, including a real robot and real-time aspects, while utilizing a simulated sensor and obstacles to enable environment randomization and automated episode resetting. We investigate what level of fine-tuning is needed for adapting to a realistic setting. Through extensive experiments, we show that our approach surpasses the performance of both previous RL-based approaches and highly specialized methods across multiple CPP variations in simulation. Meanwhile, our method successfully transfers to a real robot. Our code implementation can be found online.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods. Builds on 30 of its 65 references that are in the map. First posted on arXiv 7 Jun 2024.

Builds on

30 of 65 references.

References · 30

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  5. 2022A ConvNet for the 2020scited by 3,579
  6. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  7. 2019Searching for MobileNetV3cited by 1,994
  8. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  9. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  10. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,228
  11. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
  12. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
18 more
  1. 2020Learning to Explore using Active Neural SLAMcited by 316
  2. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 276
  3. 2018Learning to Walk via Deep Reinforcement Learningcited by 158
  4. 2021Tianshou: a Highly Modularized Deep Reinforcement Learning Librarycited by 143
  5. 2019Learning Exploration Policies for Navigationcited by 116
  6. 2018Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generationcited by 81
  7. 2019Meta Reinforcement Learning for Sim-to-real Domain Adaptationcited by 49
  8. 2020Data-efficient Domain Randomization with Bayesian Optimizationcited by 47
  9. 2019Real-Time Reinforcement Learningcited by 40
  10. 2022Asynchronous Reinforcement Learning for Real-Time Control of Physical Robotscited by 22
  11. 2020UAV Coverage Path Planning under Varying Power Constraints using Deep Reinforcement Learningcited by 15
  12. 2022Real-Time Reinforcement Learning for Vision-Based Robotics Utilizing Local and Remote Computerscited by 13
  13. 2022Explore-Bench: Data Sets, Metrics and Evaluations for Frontier-based and Deep-reinforcement-learning-based Autonomous Explorationcited by 8
  14. 2017Sim-to-Real Transfer of Accurate Grasping with Eye-In-Hand Observations and Continuous Controlcited by 7
  15. 2020Towards Closing the Sim-to-Real Gap in Collaborative Multi-Robot Deep Reinforcement Learningcited by 6
  16. 2021Reinforcement Learning-Based Coverage Path Planning with Implicit Cellular Decompositioncited by 6
  17. 2023Learning Coverage Paths in Unknown Environments with Deep Reinforcement Learningcited by 5
  18. 2018Planning Hybrid Driving-Stepping Locomotion on Multiple Levels of Abstractioncited by 2

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2024Comprehensive Overview of Reward Engineering and Shaping in Advancing Reinforcement Learning Applicationscited by 38
  2. 2025Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agentscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025Continuous World Coverage Path Planning for Fixed-Wing UAVs using Deep Reinforcement Learning
  2. 2024Deep Reinforcement Learning with Dynamic Graphs for Adaptive Informative Path Planning
  3. 2018PRIMAL: Pathfinding via Reinforcement and Imitation Multi-Agent Learning
  4. 2025Efficient Environment Design for Multi-Robot Navigation via Continuous Control ↗
  5. 2025The Emergence of Deep Reinforcement Learning for Path Planning
  6. 2026Reinforcement Learning-Based Energy-Aware Coverage Path Planning for Precision Agriculture
  7. 2025Hybrid Motion Planning with Deep Reinforcement Learning for Mobile Robot Navigation
  8. 2024OffRIPP: Offline RL-based Informative Path Planning