mlsanityData to 14 Oct 2026

Deep Reinforcement Learning for Sim-to-Real Policy Transfer of VTOL-UAVs Offshore Docking Operations

arXiv 2406.00887 ↗First posted 2 Jun 2024cs.ROcited by 4

On the map: Deep reinforcement learning methods → RL for robot navigation and UAV control

Abstract (arXiv)

This paper proposes a novel Reinforcement Learning (RL) approach for sim-to-real policy transfer of Vertical Take-Off and Landing Unmanned Aerial Vehicle (VTOL-UAV). The proposed approach is designed for VTOL-UAV landing on offshore docking stations in maritime operations. VTOL-UAVs in maritime operations encounter limitations in their operational range, primarily stemming from constraints imposed by their battery capacity. The concept of autonomous landing on a charging platform presents an intriguing prospect for mitigating these limitations by facilitating battery charging and data transfer. However, current Deep Reinforcement Learning (DRL) methods exhibit drawbacks, including lengthy training times, and modest success rates. In this paper, we tackle these concerns comprehensively by decomposing the landing procedure into a sequence of more manageable but analogous tasks in terms of an approach phase and a landing phase. The proposed architecture utilizes a model-based control scheme for the approach phase, where the VTOL-UAV is approaching the offshore docking station. In the Landing phase, DRL agents were trained offline to learn the optimal policy to dock on the offshore station. The Joint North Sea Wave Project (JONSWAP) spectrum model has been employed to create a wave model for each episode, enhancing policy generalization for sim2real transfer. A set of DRL algorithms have been tested through numerical simulations including value-based agents and policy-based agents such as Deep textitQ Networks (DQN) and Proximal Policy Optimization (PPO) respectively. The numerical experiments show that the PPO agent can learn complicated and efficient policies to land in uncertain environments, which in turn enhances the likelihood of successful sim-to-real transfer.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods. Builds on 20 of its 37 references that are in the map. First posted on arXiv 2 Jun 2024.

Builds on

20 of 37 references.

References · 20

  1. 2015Trust Region Policy Optimizationcited by 2,580
  2. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  3. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  4. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  5. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  6. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  7. 2017A Brief Survey of Deep Reinforcement Learningcited by 483
  8. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 316
  9. 2018Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applicationscited by 127
  10. 2016Sim-to-Real Robot Learning from Pixels with Progressive Netscited by 94
  11. 2017Control of a Quadrotor with Reinforcement Learningcited by 92
  12. 2018Unmanned Aerial Vehicles: A Survey on Civil Applications and Key Research Challengescited by 54
8 more
  1. 2018Reinforcement Learning for UAV Attitude Controlcited by 48
  2. 2016Survey of Important Issues in UAV Communication Networkscited by 23
  3. 2021Inclined Quadrotor Landing using Deep Reinforcement Learningcited by 7
  4. 2018Scaling simulation-to-real transfer by learning composable robot skillscited by 6
  5. 2022Exponentially Stable Observer-based Controller for VTOL-UAVs without Velocity Measurementscited by 4
  6. 2022Observer-based Controller for VTOL-UAVs Tracking using Direct Vision-Aided Inertial Navigation Measurementscited by 4
  7. 2023UWB Ranging and IMU Data Fusion: Overview and Nonlinear Stochastic Filter for Inertial Navigationcited by 4
  8. 2016Autonomous Landing of a Multirotor Micro Air Vehicle on a High Velocity Ground Vehiclecited by 2

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 2.

4 papers.

  1. 2024Comprehensive and Comparative Analysis between Transfer Learning and Custom Built VGG and CNN-SVM Models for Wildfire Detectioncited by 2
  2. 2024Novel Hybrid Integrated Pix2Pix and WGAN Model with Gradient Penalty for Binary Images Denoisingcited by 1
  3. 2025Learning to Lead Themselves: Agentic AI in MAS using MARLcited by 1
  4. 2025Novel Multi-Agent Action Masked Deep Reinforcement Learning for General Industrial Assembly Lines Balancing Problemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024The Power of Input: Benchmarking Zero-Shot Sim-To-Real Transfer of Reinforcement Learning Control Policies for Quadrotor Control
  2. 2025Transformer-Guided Deep Reinforcement Learning for Optimal Takeoff Trajectory Design of an eVTOL Drone
  3. 2025Sim-to-Real Transfer in Reinforcement Learning for Maneuver Control of a Variable-Pitch MAV
  4. 2026Sim-to-reality adaptation for Deep Reinforcement Learning applied to an underwater docking application
  5. 2025Towards Task-Oriented Flying: Framework, Infrastructure, and Principles
  6. 2026Physics-Aware End-to-End Deep Reinforcement Learning for Quadcopter Control with Actuator Dynamics
  7. 2026Robust Autonomous UAV Landing on Maritime Platforms via Multimodal Agentic AI and Active Wave Compensation
  8. 2024Tangled Program Graphs as an alternative to DRL-based control algorithms for UAVs