mlsanityData to 14 Oct 2026

Can Context Bridge the Reality Gap? Sim-to-Real Transfer of Context-Aware Policies

arXiv 2511.04249 ↗First posted 6 Nov 2025cs.ROcited by 4

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Sim-to-real transfer remains a major challenge in reinforcement learning (RL) for robotics, as policies trained in simulation often fail to generalize to the real world due to discrepancies in environment dynamics. Domain Randomization (DR) mitigates this issue by exposing the policy to a wide range of randomized dynamics during training, yet leading to a reduction in performance. While standard approaches typically train policies agnostic to these variations, we investigate whether sim-to-real transfer can be improved by conditioning the policy on an estimate of the dynamics parameters -- referred to as context. To this end, we integrate a context estimation module into a DR-based RL framework and systematically compare SOTA supervision strategies. We evaluate the resulting context-aware policies in both a canonical control benchmark and a real-world pushing task using a Franka Emika Panda robot. Results show that context-aware policies outperform the context-agnostic baseline across all settings, although the best supervision strategy depends on the task.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 20 of its 28 references that are in the map. First posted on arXiv 6 Nov 2025.

Builds on

20 of 28 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Domains as Objectives: Domain-Uncertainty-Aware Policy Optimization through Explicit Multi-Domain Convex Coverage Set Learning7 Oct

Other references · 19

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2019Solving Rubik's Cube with a Robot Handcited by 734
  4. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  5. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  6. 2023FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objectscited by 429
  7. 2015Contextual Markov Decision Processescited by 220
  8. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 170
  9. 2017Preparing for the Unknown: Learning a Universal Policy with Online System Identificationcited by 141
  10. 2018Sim-to-Real Reinforcement Learning for Deformable Object Manipulationcited by 131
  11. 2021Understanding Domain Randomization for Sim-to-real Transfercited by 80
  12. 2017Markov Decision Processes with Continuous Side Informationcited by 56
7 more
  1. 2022Contextualize Me -- The Case for Context in Reinforcement Learningcited by 46
  2. 2022Context is Everything: Implicit Identification for Dynamics Adaptationcited by 27
  3. 2023Bridging the Sim-to-Real Gap from the Information Bottleneck Perspectivecited by 10
  4. 2023Nonprehensile Planar Manipulation through Reinforcement Learning with Multimodal Categorical Explorationcited by 9
  5. 2018Sim-to-Real Transfer Learning using Robustified Controllers in Robotic Tasks involving Complex Dynamicscited by 8
  6. 2024Inferring Behavior-Specific Context Improves Zero-Shot Generalization in Reinforcement Learningcited by 5
  7. 2024Precision-Focused Reinforcement Learning Model for Robotic Object Pushingcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2026Contextual Intelligence The Next Leap for Reinforcement Learningcited by 1
  2. 2026Position: Deployed Reinforcement Learning should be Continualcited by 1
  3. 2025MagBotSim: Physics-Based Simulation and Reinforcement Learning Environments for Magnetic Roboticscited by 0
  4. 2026Dynamic Context Scheduling: Learning Beyond the Static Universecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Safe Continual Domain Adaptation after Sim2Real Transfer of Reinforcement Learning Policies in Robotics
  2. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey
  3. 2025Revealing the Challenges of Sim-to-Real Transfer in Model-Based Reinforcement Learning via Latent Space Modeling
  4. 2025CARoL: Context-aware Adaptation for Robot Learning
  5. 2025An Real-Sim-Real (RSR) Loop Framework for Generalizable Robotic Policy Transfer
  6. 2026Support-Constrained RL Enables Real-World Policy Improvement without Real-World Experience
  7. 2026Abstract Sim2Real through Approximate Information States
  8. 2025SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfer