# Shadow: Leveraging Segmentation Masks for Cross-Embodiment Policy Transfer

arXiv: https://arxiv.org/abs/2503.00774 · HTML: https://mlsanity.com/paper/2503.00774/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Mar 2025.

## Built on by (most cited first, 14 of 14)

- [X-Sim: Cross-Embodiment Learning via Real-to-Sim-to-Real](https://mlsanity.com/paper/2505.07096/) (2025)
- [MV-UMI: A Scalable Multi-View Interface for Cross-Embodiment Learning](https://mlsanity.com/paper/2509.18757/) (2025)
- [H2R-Grounder: A Paired-Data-Free Paradigm for Translating Human Interaction Videos into Physically Grounded Robot Videos](https://mlsanity.com/paper/2512.09406/) (2025)
- [Masked Visual Actions for Unified World Modeling](https://mlsanity.com/paper/2607.19343/) (2026)
- [Robot Learning from Any Images](https://mlsanity.com/paper/2509.22970/) (2025)
- [X-Diffusion: Training Diffusion Policies on Cross-Embodiment Human Demonstrations](https://mlsanity.com/paper/2511.04671/) (2025)
- [HandEdit: A Unified Benchmark for Egocentric Human-to-Robot Dexterous Hand Image Editing](https://mlsanity.com/paper/2608.12122/) (2026)
- [CRAFT: Video Diffusion for Bimanual Robot Data Generation](https://mlsanity.com/paper/2604.03552/) (2026)
- [Cross-Embodiment Transfer via Behavior-Aligned Representations](https://mlsanity.com/paper/2607.27549/) (2026)
- [The Embodiment Gap in Robot Foundation Models](https://mlsanity.com/paper/2608.18433/) (2026)
- [SegDAC: Visual Generalization in Reinforcement Learning via Dynamic Object Tokens](https://mlsanity.com/paper/2508.09325/) (2025)
- [GORDON: Graph-based Object-centric Rewards for Decomposition of Long-Horizon Manipulation](https://mlsanity.com/paper/2608.03753/) (2026)
- [ZETA: A Controlled Study of Zero-Shot Cross-Embodiment VLA Transfer for Tabletop Manipulation](https://mlsanity.com/paper/2609.02546/) (2026)
- [X-Reset: Scaling Object-Centric Reinforcement Learning via Cross-Embodiment Resets](https://mlsanity.com/paper/2609.35715/) (2026)

## Abstract (arXiv)

Data collection in robotics is spread across diverse hardware, and this variation will increase as new hardware is developed. Effective use of this growing body of data requires methods capable of learning from diverse robot embodiments. We consider the setting of training a policy using expert trajectories from a single robot arm (the source), and evaluating on a different robot arm for which no data was collected (the target). We present a data editing scheme termed Shadow, in which the robot during training and evaluation is replaced with a composite segmentation mask of the source and target robots. In this way, the input data distribution at train and test time match closely, enabling robust policy transfer to the new unseen robot while being far more data efficient than approaches that require co-training on large amounts of data from diverse embodiments. We demonstrate that an approach as simple as Shadow is effective both in simulation on varying tasks and robots, and on real robot hardware, where Shadow demonstrates an average of over 2x improvement in success rate compared to the strongest baseline.