Dual Action Policy for Robust Sim-to-Real Reinforcement Learning
On the map: Deep reinforcement learning methods → Safe and constrained RL
Abstract (arXiv)
This paper presents Dual Action Policy (DAP), a novel approach to address the dynamics mismatch inherent in the sim-to-real gap of reinforcement learning. DAP uses a single policy to predict two sets of actions: one for maximizing task rewards in simulation and another specifically for domain adaptation via reward adjustments. This decoupling makes it easier to maximize the overall reward in the source domain during training. Additionally, DAP incorporates uncertainty-based exploration during training to enhance agent robustness. Experimental results demonstrate DAP's effectiveness in bridging the sim-to-real gap, outperforming baselines on challenging tasks in simulation, and further improvement is achieved by incorporating uncertainty estimation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 21 of its 29 references that are in the map. First posted on arXiv 16 Oct 2024.
Builds on
21 of 29 references.
References · 21
- 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
- 2015Continuous control with deep reinforcement learningcited by 3,407
- 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
- 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
- 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
- 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
- 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
- 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
- 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 353
- 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemblecited by 302
- 2018Model-Ensemble Trust-Region Policy Optimizationcited by 219
- 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 217
9 more
- 2017Preparing for the Unknown: Learning a Universal Policy with Online System Identificationcited by 141
- 2020Off-Dynamics Reinforcement Learning: Training for Transfer with Domain Classifierscited by 97
- 2022Why So Pessimistic? Estimating Uncertainties for Offline RL through Ensembles, and Why Their Independence Matterscited by 76
- 2022Reward Uncertainty for Exploration in Preference-based Reinforcement Learningcited by 61
- 2022When to Trust Your Simulator: Dynamics-Aware Hybrid Offline-and-Online Reinforcement Learningcited by 60
- 2020Reinforcement Learning in Economics and Financecited by 58
- 2016Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Modelcited by 54
- 2021SimGAN: Hybrid Simulator Identification for Domain Adaptation via Adversarial Reinforcement Learningcited by 27
- 2020Reinforced Grounded Action Transformation for Sim-to-Real Transfercited by 18
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Theoretical Foundations and Effective Algorithms for Policy-Aware Simulator Learning
- 2024Constrained Reinforcement Learning Under Model Mismatch
- 2024Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL
- 2026Robust Peak-cost Constrained Reinforcement Learning
- 2025Control Synthesis with Reinforcement Learning: A Modeling Perspective
- 2024Safe and Robust Reinforcement Learning: Principles and Practice
- 2025Safe But Not Sorry: Reducing Over-Conservatism in Safety Critics via Uncertainty-Aware Modulation
- 2025Action Robust Reinforcement Learning via Optimal Adversary Aware Policy Optimization