mlsanityData to 14 Oct 2026

Abstract Sim2Real through Approximate Information States

arXiv 2604.15289 ↗First posted 16 Apr 2026cs.ROcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

In recent years, reinforcement learning (RL) has shown remarkable success in robotics when a fast and accurate simulator is available for a given task. When using RL and simulation, more simulator realism is generally beneficial but becomes harder to obtain as robots are deployed in increasingly complex and widescale domains. In such settings, simulators will likely fail to model all relevant details of a given target task and this observation motivates the study of sim2real with simulators that leave out key task details. In this paper, we formalize and study the abstract sim2real problem: given an abstract simulator that models a target task at a coarse level of abstraction, how can we train a policy with RL in the abstract simulator and successfully transfer it to the real-world? Our first contribution is to formalize this problem using the language of state abstraction from the RL literature. This framing shows that an abstract simulator can be grounded to match the target task if the grounded abstract dynamics take the history of states into account. Based on the formalism, we then introduce a method that uses real-world task data to correct the dynamics of the abstract simulator. We then show that this method enables successful policy transfer both in sim2sim and sim2real evaluation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 31 of its 41 references that are in the map. First posted on arXiv 16 Apr 2026.

Builds on

31 of 41 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Reinforcement Learning Within the Classical Robotics Stack: A Case Study in Robot Soccer12 Dec

Other references · 30

  1. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  2. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  3. 2018Learning Dexterous In-Hand Manipulationcited by 687
  4. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  5. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
  6. 2019Learning agile and dynamic motor skills for legged robotscited by 504
  7. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 465
  8. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
  9. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 371
  10. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 362
  11. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 319
  12. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
18 more
  1. 2023Real-World Humanoid Locomotion with Reinforcement Learningcited by 187
  2. 2017Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Graspingcited by 144
  3. 2018Driving Policy Transfer via Modularity and Abstractioncited by 92
  4. 2022BYOL-Explore: Exploration by Bootstrapped Predictioncited by 73
  5. 2020Approximate information state for approximate planning and reinforcement learning in partially observed systemscited by 70
  6. 2021People construct simplified mental representations to plancited by 47
  7. 2021Learning Markov State Abstractions for Deep Reinforcement Learningcited by 44
  8. 2019Multi-Agent Manipulation via Locomotion using Hierarchical Sim2Realcited by 42
  9. 2022Rethinking Sim2Real: Lower Fidelity Simulation Leads to Higher Sim2Real Transfer in Navigationcited by 30
  10. 2022A Theory of Abstraction in Reinforcement Learningcited by 27
  11. 2023What Went Wrong? Closing the Sim-to-Real Gap via Differentiable Causal Discoverycited by 27
  12. 2020Perspectives on Sim2Real Transfer for Robotics: A Summary of the R:SS 2020 Workshopcited by 25
  13. 2021GridToPix: Training Embodied Agents with Minimal Supervisioncited by 21
  14. 2020Reinforced Grounded Action Transformation for Sim-to-Real Transfercited by 18
  15. 2022On learning history based policies for controlling Markov decision processescited by 5
  16. 2024Abstract Reward Processes: Leveraging State Abstraction for Consistent Off-Policy Evaluationcited by 3
  17. 2025From Abstraction to Reality: DARPA's Vision for Robust Sim-to-Real Autonomycited by 3
  18. 2025Multi-Robot Collaboration through Reinforcement Learning and Abstract Simulationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Overcoming the Sim-to-Real Gap: Leveraging Simulation to Learn to Explore for Real-World RL
  2. 2025An Real-Sim-Real (RSR) Loop Framework for Generalizable Robotic Policy Transfer
  3. 2025A Survey of Sim-to-Real Methods in RL: Progress, Prospects and Challenges with Foundation Models
  4. 2023H2O+: An Improved Framework for Hybrid Offline-and-Online RL with Dynamics Gaps
  5. 2025SimLauncher: Launching Sample-Efficient Real-world Robotic Reinforcement Learning via Simulation Pre-training
  6. 2025Can Context Bridge the Reality Gap? Sim-to-Real Transfer of Context-Aware Policies
  7. 2025Revealing the Challenges of Sim-to-Real Transfer in Model-Based Reinforcement Learning via Latent Space Modeling
  8. 2026A Balanced Data Diet: Addressing the Exploration Bottleneck in Mega-Scale RL for Robot Control