# Sim-to-Real Transfer and Robustness Evaluation of Reinforcement Learning Control with Integrated Perception on an ASV for Floating Waste Capture

arXiv: https://arxiv.org/abs/2605.02529 · HTML: https://mlsanity.com/paper/2605.02529/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 22 of its 63 references that are in the map.
- First posted on arXiv 4 May 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 22 of 22)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Object Detection in 20 Years: A Survey](https://mlsanity.com/paper/1905.05055/) (2019)
- [Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments](https://mlsanity.com/paper/2301.04195/) (2023)
- [Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations](https://mlsanity.com/paper/2003.08938/) (2020)
- [Real-World Humanoid Locomotion with Reinforcement Learning](https://mlsanity.com/paper/2303.03381/) (2023)
- [ANYmal Parkour: Learning Agile Navigation for Quadrupedal Robots](https://mlsanity.com/paper/2306.14874/) (2023)
- [How Simulation Helps Autonomous Driving:A Survey of Sim2real, Digital Twins, and Parallel Intelligence](https://mlsanity.com/paper/2305.01263/) (2023)
- [A Review of Uncertainty for Deep Reinforcement Learning](https://mlsanity.com/paper/2208.09052/) (2022)
- [A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Models](https://mlsanity.com/paper/2502.13187/) (2025)
- [Model-Reference Reinforcement Learning for Collision-Free Tracking Control of Autonomous Surface Vehicles](https://mlsanity.com/paper/2008.07240/) (2020)
- [Survey of Deep Learning for Autonomous Surface Vehicles in the Marine Environment](https://mlsanity.com/paper/2210.08487/) (2022)
- [Robust Unmanned Surface Vehicle Navigation with Distributional Reinforcement Learning](https://mlsanity.com/paper/2307.16240/) (2023)
- [Deep Reinforcement Learning Based Tracking Control of an Autonomous Surface Vessel in Natural Waters](https://mlsanity.com/paper/2302.08100/) (2023)
- [A Deep Reinforcement Learning Framework and Methodology for Reducing the Sim-to-Real Gap in ASV Navigation](https://mlsanity.com/paper/2407.08263/) (2024)
- [Optimizing Plastic Waste Collection in Water Bodies Using Heterogeneous Autonomous Surface Vehicles with Deep Reinforcement Learning](https://mlsanity.com/paper/2412.02316/) (2024)
- [PoTATO: A Dataset for Analyzing Polarimetric Traces of Afloat Trash Objects](https://mlsanity.com/paper/2409.12659/) (2024)
- [An Informative Planning Framework for Target Tracking and Active Mapping in Dynamic Environments with ASVs](https://mlsanity.com/paper/2508.14636/) (2025)
- [Field Testing of a Stochastic Planner for ASV Navigation Using Satellite Images](https://mlsanity.com/paper/2309.14657/) (2023)
- [Evaluating Robustness of Deep Reinforcement Learning for Autonomous Surface Vehicle Control in Field Tests](https://mlsanity.com/paper/2505.10033/) (2025)
- [Evaluation of Polarimetric Fusion for Semantic Segmentation in Aquatic Environments](https://mlsanity.com/paper/2509.24731/) (2025)

## Abstract (arXiv)

Autonomous surface vessels for floating-waste removal operate under varying hydrodynamics, external disturbances, and challenging water-surface perception. We present a field-validated system that combines camera-based polarimetric perception with a lightweight DRL-based controller for floating-waste detection and capture. Camera detections are converted into water-surface target points and tracked by a controller trained entirely in simulation and deployed directly on a retrofitted ASV platform. Our main contribution is a sim-to-real testing methodology that combines a two-stage simulation protocol with a perception abstraction module designed to mimic real camera behavior, enabling reproducible field trials and explicit evaluation of the sim-to-real gap. We apply this framework in matched simulation and field experiments across 14 disturbance regimes to expose failure modes and evaluate robustness. The results show centimeter-level terminal accuracy and indicate robust control performance under the evaluated perturbation regimes. The main source of degradation is insufficient actuation-model fidelity. We also demonstrate the system in a search-and-capture application using real camera detections in real-world conditions over areas of up to 450~m2. The study distills practical lessons for reliable transfer, including improved actuation-model fidelity, targeted domain randomization, and careful management of latency and timestamps across modules, while highlighting remaining challenges.