# Can Context Bridge the Reality Gap? Sim-to-Real Transfer of Context-Aware Policies

arXiv: https://arxiv.org/abs/2511.04249 · HTML: https://mlsanity.com/paper/2511.04249/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 20 of its 28 references that are in the map.
- First posted on arXiv 6 Nov 2025.

## Placed in

- Deep reinforcement learning methods / [Safe and constrained RL](https://mlsanity.com/deeprl/safe-and-constrained-rl/)

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects](https://mlsanity.com/paper/2312.08344/) (2023)
- [Contextual Markov Decision Processes](https://mlsanity.com/paper/1502.02259/) (2015)
- [A Survey of Zero-shot Generalisation in Deep Reinforcement Learning](https://mlsanity.com/paper/2111.09794/) (2021)
- [Preparing for the Unknown: Learning a Universal Policy with Online System Identification](https://mlsanity.com/paper/1702.02453/) (2017)
- [Sim-to-Real Reinforcement Learning for Deformable Object Manipulation](https://mlsanity.com/paper/1806.07851/) (2018)
- [Understanding Domain Randomization for Sim-to-real Transfer](https://mlsanity.com/paper/2110.03239/) (2021)
- [Markov Decision Processes with Continuous Side Information](https://mlsanity.com/paper/1711.05726/) (2017)
- [Contextualize Me -- The Case for Context in Reinforcement Learning](https://mlsanity.com/paper/2202.04500/) (2022)
- [Context is Everything: Implicit Identification for Dynamics Adaptation](https://mlsanity.com/paper/2203.05549/) (2022)
- [Bridging the Sim-to-Real Gap from the Information Bottleneck Perspective](https://mlsanity.com/paper/2305.18464/) (2023)
- [Nonprehensile Planar Manipulation through Reinforcement Learning with Multimodal Categorical Exploration](https://mlsanity.com/paper/2308.02459/) (2023)
- [Sim-to-Real Transfer Learning using Robustified Controllers in Robotic Tasks involving Complex Dynamics](https://mlsanity.com/paper/1809.04720/) (2018)
- [Inferring Behavior-Specific Context Improves Zero-Shot Generalization in Reinforcement Learning](https://mlsanity.com/paper/2404.09521/) (2024)
- [Precision-Focused Reinforcement Learning Model for Robotic Object Pushing](https://mlsanity.com/paper/2411.08622/) (2024)
- [Domains as Objectives: Domain-Uncertainty-Aware Policy Optimization through Explicit Multi-Domain Convex Coverage Set Learning](https://mlsanity.com/paper/2410.04719/) (2024)

## Built on by (most cited first, 4 of 4)

- [Contextual Intelligence The Next Leap for Reinforcement Learning](https://mlsanity.com/paper/2604.02348/) (2026)
- [Position: Deployed Reinforcement Learning should be Continual](https://mlsanity.com/paper/2606.04029/) (2026)
- [MagBotSim: Physics-Based Simulation and Reinforcement Learning Environments for Magnetic Robotics](https://mlsanity.com/paper/2511.16158/) (2025)
- [Dynamic Context Scheduling: Learning Beyond the Static Universe](https://mlsanity.com/paper/2608.20799/) (2026)

## Abstract (arXiv)

Sim-to-real transfer remains a major challenge in reinforcement learning (RL) for robotics, as policies trained in simulation often fail to generalize to the real world due to discrepancies in environment dynamics. Domain Randomization (DR) mitigates this issue by exposing the policy to a wide range of randomized dynamics during training, yet leading to a reduction in performance. While standard approaches typically train policies agnostic to these variations, we investigate whether sim-to-real transfer can be improved by conditioning the policy on an estimate of the dynamics parameters -- referred to as context. To this end, we integrate a context estimation module into a DR-based RL framework and systematically compare SOTA supervision strategies. We evaluate the resulting context-aware policies in both a canonical control benchmark and a real-world pushing task using a Franka Emika Panda robot. Results show that context-aware policies outperform the context-agnostic baseline across all settings, although the best supervision strategy depends on the task.