# NavRL++: A System-Level Framework for Improving Sim-to-Real Transfer in Reinforcement Learning-Based Robot Navigation arXiv: https://arxiv.org/abs/2605.15559 ยท HTML: https://mlsanity.com/paper/2605.15559/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods. - Builds on 47 of its 71 references that are in the map. - First posted on arXiv 15 May 2026. ## Placed in - Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/) ## Builds on (most cited first, 30 of 47) - [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017) - [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015) - [VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator](https://mlsanity.com/paper/1708.03852/) (2017) - [Benchmarking Deep Reinforcement Learning for Continuous Control](https://mlsanity.com/paper/1604.06778/) (2016) - [NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration](https://mlsanity.com/paper/2310.07896/) (2023) - [CAD2RL: Real Single-Image Flight without a Single Real Image](https://mlsanity.com/paper/1611.04201/) (2016) - [ViNT: A Foundation Model for Visual Navigation](https://mlsanity.com/paper/2306.14846/) (2023) - [FAST-LIO2: Fast Direct LiDAR-inertial Odometry](https://mlsanity.com/paper/2107.06829/) (2021) - [Socially Aware Motion Planning with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.08862/) (2017) - [Learning High-Speed Flight in the Wild](https://mlsanity.com/paper/2110.05113/) (2021) - [Decentralized Non-communicating Multiagent Collision Avoidance with Deep Reinforcement Learning](https://mlsanity.com/paper/1609.07845/) (2016) - [Agile But Safe: Learning Collision-Free High-Speed Legged Locomotion](https://mlsanity.com/paper/2401.17583/) (2024) - [TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction](https://mlsanity.com/paper/2405.10315/) (2024) - [EGO-Planner: An ESDF-free Gradient-based Local Planner for Quadrotors](https://mlsanity.com/paper/2008.08835/) (2020) - [Collision Avoidance in Pedestrian-Rich Environments with Deep Reinforcement Learning](https://mlsanity.com/paper/1910.11689/) (2019) - [Back to Newton's Laws: Learning Vision-based Agile Flight via Differentiable Physics](https://mlsanity.com/paper/2407.10648/) (2024) - [Learning to Fly by Crashing](https://mlsanity.com/paper/1704.05588/) (2017) - [Robust and Efficient Quadrotor Trajectory Generation for Fast Autonomous Flight](https://mlsanity.com/paper/1907.01531/) (2019) - [Learning a State Representation and Navigation in Cluttered and Dynamic Environments](https://mlsanity.com/paper/2103.04351/) (2021) - [DRL-VO: Learning to Navigate Through Crowded Dynamic Scenes Using Velocity Obstacles](https://mlsanity.com/paper/2301.06512/) (2023) - [Memory-based Deep Reinforcement Learning for Obstacle Avoidance in UAV with Limited Environment Knowledge](https://mlsanity.com/paper/1811.03307/) (2018) - [Cooperative Multi-Agent Deep Reinforcement Learning for Reliable Surveillance via Autonomous Multi-UAV Control](https://mlsanity.com/paper/2201.05843/) (2022) - [Sim-To-Real Transfer for Visual Reinforcement Learning of Deformable Object Manipulation for Robot-Assisted Surgery](https://mlsanity.com/paper/2406.06092/) (2024) - [NavRL: Learning Safe Flight in Dynamic Environments](https://mlsanity.com/paper/2409.15634/) (2024) - [Towards Monocular Vision based Obstacle Avoidance through Deep Reinforcement Learning](https://mlsanity.com/paper/1706.09829/) (2017) - [Learning Perception-Aware Agile Flight in Cluttered Environments](https://mlsanity.com/paper/2210.01841/) (2022) - [Toward Low-Flying Autonomous MAV Trail Navigation using Deep Neural Networks for Environmental Awareness](https://mlsanity.com/paper/1705.02550/) (2017) - [DROPO: Sim-to-Real Transfer with Offline Domain Randomization](https://mlsanity.com/paper/2201.08434/) (2022) - [Onboard dynamic-object detection and tracking for autonomous robot navigation with RGB-D camera](https://mlsanity.com/paper/2303.00132/) (2023) - [Contrastive Learning for Enhancing Robust Scene Transfer in Vision-based Agile Flight](https://mlsanity.com/paper/2309.09865/) (2023) ## Built on by (most cited first, 1 of 1) - [IR-SIM: A Lightweight Declarative Simulator for Navigation Learning and Benchmarking](https://mlsanity.com/paper/2606.08729/) (2026) ## Abstract (arXiv) Recent years have witnessed significant progress in autonomous navigation using reinforcement learning. However, existing approaches largely emphasize reinforcement learning framework design, such as input representations, action spaces, and reward functions, while providing limited analysis of sim-to-real transfer and insufficient insight into how training strategies affect real-world deployment performance. To bridge this gap, we not only introduce an effective RL framework but also present a complete training and deployment pipeline, along with a systematic empirical study that disentangles the key factors affecting sim-to-real transfer in reinforcement learning-based navigation, including sensor noise, perception failures, system latency, and control response. Building on insights from this analysis, we introduce perturbation-aware fine-tuning, a post-training adaptation strategy that improves transfer robustness by explicitly accounting for empirically identified domain discrepancies. To further mitigate perception degradation and enhance control smoothness in real-world deployment, we propose a Transformer-based temporal reasoning policy that leverages short-horizon observation for navigation control. We quantitatively evaluate how individual sim-to-real perturbations and training design choices impact navigation performance across environments. Experimental results demonstrate that the proposed training strategy and policy architecture outperform learning-based baselines in both static and dynamic environments, while achieving performance comparable to optimization-based planners in static settings. We validate our approach through real-world deployment on multiple robotic platforms, including aerial and legged robots, across navigation-centric tasks such as exploration and inspection, demonstrating zero-shot sim-to-real transfer.