# Adapting Deep Visuomotor Representations with Weak Pairwise Constraints

arXiv: https://arxiv.org/abs/1511.07111 · HTML: https://mlsanity.com/paper/1511.07111/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Nov 2015.

## Built on by (most cited first, 25 of 25)

- [DayDreamer: World Models for Physical Robot Learning](https://mlsanity.com/paper/2206.14176/) (2022)
- [CACTI: A Framework for Scalable Multi-Task Multi-Scene Visual Imitation Learning](https://mlsanity.com/paper/2212.05711/) (2022)
- [How Simulation Helps Autonomous Driving:A Survey of Sim2real, Digital Twins, and Parallel Intelligence](https://mlsanity.com/paper/2305.01263/) (2023)
- [Domain Adaptation In Reinforcement Learning Via Latent Unified State Representation](https://mlsanity.com/paper/2102.05714/) (2021)
- [SIMS-V: Simulated Instruction-Tuning for Spatial Video Understanding](https://mlsanity.com/paper/2511.04668/) (2025)
- [Weakly Supervised Correspondence Learning](https://mlsanity.com/paper/2203.00904/) (2022)
- [DiAReL: Reinforcement Learning with Disturbance Awareness for Robust Sim2Real Policy Transfer in Robot Control](https://mlsanity.com/paper/2306.09010/) (2023)
- [Transport Discrepancy as a Reliability Signal for Vision-Language-Action Models](https://mlsanity.com/paper/2512.01715/) (2025)
- [Learning from Imperfect Demonstrations via Adversarial Confidence Transfer](https://mlsanity.com/paper/2202.02967/) (2022)
- [Multi-Environment Pretraining Enables Transfer to Action Limited Datasets](https://mlsanity.com/paper/2211.13337/) (2022)
- [Online Prototype Alignment for Few-shot Policy Transfer](https://mlsanity.com/paper/2306.07307/) (2023)
- [Learning Deep Features for Shape Correspondence with Domain Invariance](https://mlsanity.com/paper/2102.10493/) (2021)
- [Domain Adaptive Egocentric Person Re-identification](https://mlsanity.com/paper/2103.04870/) (2021)
- [SIM2REALVIZ: Visualizing the Sim2Real Gap in Robot Ego-Pose Estimation](https://mlsanity.com/paper/2109.11801/) (2021)
- [HaDR: Applying Domain Randomization for Generating Synthetic Multimodal Dataset for Hand Instance Segmentation in Cluttered Industrial Environments](https://mlsanity.com/paper/2304.05826/) (2023)
- [Building Manufacturing Deep Learning Models with Minimal and Imbalanced Training Data Using Domain Adaptation and Data Augmentation](https://mlsanity.com/paper/2306.00202/) (2023)
- [Off Environment Evaluation Using Convex Risk Minimization](https://mlsanity.com/paper/2112.11532/) (2021)
- [Machine Learning Meets Advanced Robotic Manipulation](https://mlsanity.com/paper/2309.12560/) (2023)
- [cMALC-D: Contextual Multi-Agent LLM-Guided Curriculum Learning with Diversity-Based Context Blending](https://mlsanity.com/paper/2508.20818/) (2025)
- [Training and Evaluation of Deep Policies using Reinforcement Learning and Generative Models](https://mlsanity.com/paper/2204.08573/) (2022)
- [Domain Adaptation via Rebalanced Sub-domain Alignment](https://mlsanity.com/paper/2302.02009/) (2023)
- [Controlling by Showing: i-Mimic: A Video-based Method to Control Robotic Arms](https://mlsanity.com/paper/2101.11451/) (2021)
- [On Assessing the Usefulness of Proxy Domains for Developing and Evaluating Embodied Agents](https://mlsanity.com/paper/2109.14516/) (2021)
- [Few-Shot Image-to-Semantics Translation for Policy Transfer in Reinforcement Learning](https://mlsanity.com/paper/2301.13343/) (2023)
- [Sim-to-Real Deep Reinforcement Learning with Manipulators for Pick-and-place](https://mlsanity.com/paper/2309.09247/) (2023)

## Abstract (arXiv)

Real-world robotics problems often occur in domains that differ significantly from the robot's prior training environment. For many robotic control tasks, real world experience is expensive to obtain, but data is easy to collect in either an instrumented environment or in simulation. We propose a novel domain adaptation approach for robot perception that adapts visual representations learned on a large easy-to-obtain source dataset (e.g. synthetic images) to a target real-world domain, without requiring expensive manual data annotation of real world data before policy search. Supervised domain adaptation methods minimize cross-domain differences using pairs of aligned images that contain the same object or scene in both the source and target domains, thus learning a domain-invariant representation. However, they require manual alignment of such image pairs. Fully unsupervised adaptation methods rely on minimizing the discrepancy between the feature distributions across domains. We propose a novel, more powerful combination of both distribution and pairwise image alignment, and remove the requirement for expensive annotation by using weakly aligned pairs of images in the source and target domains. Focusing on adapting from simulation to real world data using a PR2 robot, we evaluate our approach on a manipulation task and show that by using weakly paired images, our method compensates for domain shift more effectively than previous techniques, enabling better robot performance in the real world.