# DexStitch: Learning and Composing Dexterous Skills from Human Videos via Contact-Aware Robotic Inpainting

arXiv: https://arxiv.org/abs/2610.17789 · HTML: https://mlsanity.com/paper/2610.17789/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 35 of its 38 references that are in the map.
- First posted on arXiv 13 Oct 2026.

## Builds on (most cited first, 30 of 35)

- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning](https://mlsanity.com/paper/2506.09985/) (2025)
- [R3M: A Universal Visual Representation for Robot Manipulation](https://mlsanity.com/paper/2203.12601/) (2022)
- [Learning Latent Plans from Play](https://mlsanity.com/paper/1903.01973/) (2019)
- [MimicPlay: Long-Horizon Imitation Learning by Watching Human Play](https://mlsanity.com/paper/2302.12422/) (2023)
- [ProPainter: Improving Propagation and Transformer for Video Inpainting](https://mlsanity.com/paper/2309.03897/) (2023)
- [EgoMimic: Scaling Imitation Learning via Egocentric Video](https://mlsanity.com/paper/2410.24221/) (2024)
- [EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos](https://mlsanity.com/paper/2507.12440/) (2025)
- [DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm System](https://mlsanity.com/paper/1910.03135/) (2019)
- [XIRL: Cross-embodiment Inverse Reinforcement Learning](https://mlsanity.com/paper/2106.03911/) (2021)
- [EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data](https://mlsanity.com/paper/2602.16710/) (2026)
- [Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulation](https://mlsanity.com/paper/2109.13841/) (2021)
- [Humanoid Policy ~ Human Policy](https://mlsanity.com/paper/2503.13441/) (2025)
- [XSkill: Cross Embodiment Skill Discovery](https://mlsanity.com/paper/2307.09955/) (2023)
- [Phantom: Training Robots Without Robots Using Only Human Videos](https://mlsanity.com/paper/2503.00779/) (2025)
- [LOTUS: Continual Imitation Learning for Robot Manipulation Through Unsupervised Skill Discovery](https://mlsanity.com/paper/2311.02058/) (2023)
- [HaWoR: World-Space Hand Motion Reconstruction from Egocentric Videos](https://mlsanity.com/paper/2501.02973/) (2025)
- [Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulation](https://mlsanity.com/paper/2309.00987/) (2023)
- [Adversarial Skill Chaining for Long-Horizon Robot Manipulation via Terminal State Regularization](https://mlsanity.com/paper/2111.07999/) (2021)
- [Masquerade: Learning from In-the-wild Human Videos using Data-Editing](https://mlsanity.com/paper/2508.09976/) (2025)
- [DexMachina: Functional Retargeting for Bimanual Dexterous Manipulation](https://mlsanity.com/paper/2505.24853/) (2025)
- [UniSkill: Imitating Human Videos via Cross-Embodiment Skill Representations](https://mlsanity.com/paper/2505.08787/) (2025)
- [H2R: A Human-to-Robot Data Augmentation for Robot Pre-training from Videos](https://mlsanity.com/paper/2505.11920/) (2025)
- [EasyHeC: Accurate and Automatic Hand-eye Calibration via Differentiable Rendering and Space Exploration](https://mlsanity.com/paper/2305.01191/) (2023)
- [Learning Dense Hand Contact Estimation from Imbalanced Data](https://mlsanity.com/paper/2505.11152/) (2025)
- [LiLo-VLA: Compositional Long-Horizon Manipulation via Linked Object-Centric Policies](https://mlsanity.com/paper/2602.21531/) (2026)

## Abstract (arXiv)

A long-horizon manipulation task consists of primitive skills and the transitions connecting them. Learning skills that can be executed in sequence requires supervision for both, which is costly to collect on a robot. Human videos offer a scalable source of demonstrations, but transferring them to robots requires extracting robot-compatible observations and actions together with their skill structure. We present DexStitch, a framework that cuts long-horizon human videos into skills and transitions, converts them into aligned robot observation-action-skill data, and stitches the learned skills into a requested sequence. A skill classifier first segments the videos into manipulation skills and transitions. DexStitch then converts the human demonstrations into robot observations and actions while retaining their corresponding skill labels. During conversion, per-finger human-object contacts guide dexterous hand retargeting and robotic inpainting, preserving contact geometry and finger-object occlusion ordering. To stitch the learned skills into a sequence, a shared skill-conditioned policy is pretrained on the resulting synthetic data and finetuned on a mixture of synthetic data and a small set of single-skill teleoperation episodes. Conditioning on the current and next skills guides each transition toward the following skill, while predicted skill progression enables closed-loop switching. On a held-out session, the classifier agrees with manual annotations on 92.4% of frames. Across four household skills, DexStitch achieves 80.0% single-skill success and 77.5% sequence success.