# Self-Supervised Sim-to-Real Adaptation for Visual Robotic Manipulation

arXiv: https://arxiv.org/abs/1910.09470 · HTML: https://mlsanity.com/paper/1910.09470/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Oct 2019.

## Built on by (most cited first, 19 of 19)

- [Prototypical Cross-domain Self-supervised Learning for Few-shot Unsupervised Domain Adaptation](https://mlsanity.com/paper/2103.16765/) (2021)
- [Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapes](https://mlsanity.com/paper/2110.06192/) (2021)
- [Auto-Tuned Sim-to-Real Transfer](https://mlsanity.com/paper/2104.07662/) (2021)
- [Lossless Adaptation of Pretrained Vision Models For Robotic Manipulation](https://mlsanity.com/paper/2304.06600/) (2023)
- [Policy Transfer across Visual and Dynamics Domain Gaps via Iterative Grounding](https://mlsanity.com/paper/2107.00339/) (2021)
- [Distilling Motion Planner Augmented Policies into Visual Control Policies for Robot Manipulation](https://mlsanity.com/paper/2111.06383/) (2021)
- [Multi-source Few-shot Domain Adaptation](https://mlsanity.com/paper/2109.12391/) (2021)
- [Real-to-Sim: Predicting Residual Errors of Robotic Systems with Sparse Data using a Learning-based Unscented Kalman Filter](https://mlsanity.com/paper/2209.03210/) (2022)
- [ADPro: a Test-time Adaptive Diffusion Policy via Manifold-constrained Denoising and Task-aware Initialization for Robotic Manipulation](https://mlsanity.com/paper/2508.06266/) (2025)
- [Embodied Co-Design for Rapidly Evolving Agents: Taxonomy, Frontiers, and Challenges](https://mlsanity.com/paper/2512.04770/) (2025)
- [Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Review](https://mlsanity.com/paper/2411.10268/) (2024)
- [Learning to Execute: Efficient Learning of Universal Plan-Conditioned Policies in Robotics](https://mlsanity.com/paper/2111.07908/) (2021)
- [Learning Sim-to-Real Dense Object Descriptors for Robotic Manipulation](https://mlsanity.com/paper/2304.08703/) (2023)
- [On the Challenges of Open World Recognitionunder Shifting Visual Domains](https://mlsanity.com/paper/2107.04461/) (2021)
- [Tactile Estimation of Extrinsic Contact Patch for Stable Placement](https://mlsanity.com/paper/2309.14552/) (2023)
- [Contrastive Learning for Cross-Domain Open World Recognition](https://mlsanity.com/paper/2203.09257/) (2022)
- [A Q-learning approach to the continuous control problem of robot inverted pendulum balancing](https://mlsanity.com/paper/2312.02649/) (2023)
- [Sim-to-Real Transfer via a Style-Identified Cycle Consistent Generative Adversarial Network: Zero-Shot Deployment on Robotic Manipulators through Visual Domain Adaptation](https://mlsanity.com/paper/2601.16677/) (2026)
- [A Comprehensive Review of Generative Physical Artificial Intelligence](https://mlsanity.com/paper/2609.18111/) (2026)

## Abstract (arXiv)

Collecting and automatically obtaining reward signals from real robotic visual data for the purposes of training reinforcement learning algorithms can be quite challenging and time-consuming. Methods for utilizing unlabeled data can have a huge potential to further accelerate robotic learning. We consider here the problem of performing manipulation tasks from pixels. In such tasks, choosing an appropriate state representation is crucial for planning and control. This is even more relevant with real images where noise, occlusions and resolution affect the accuracy and reliability of state estimation. In this work, we learn a latent state representation implicitly with deep reinforcement learning in simulation, and then adapt it to the real domain using unlabeled real robot data. We propose to do so by optimizing sequence-based self supervised objectives. These exploit the temporal nature of robot experience, and can be common in both the simulated and real domains, without assuming any alignment of underlying states in simulated and unlabeled real images. We propose Contrastive Forward Dynamics loss, which combines dynamics model learning with time-contrastive techniques. The learned state representation that results from our methods can be used to robustly solve a manipulation task in simulation and to successfully transfer the learned skill on a real system. We demonstrate the effectiveness of our approaches by training a vision-based reinforcement learning agent for cube stacking. Agents trained with our method, using only 5 hours of unlabeled real robot data for adaptation, shows a clear improvement over domain randomization, and standard visual domain adaptation techniques for sim-to-real transfer.