# LUCID: Learning Embodiment-Agnostic Intent Models from Unstructured Human Videos for Scalable Dexterous Robot Skill Acquisition

arXiv: https://arxiv.org/abs/2606.11628 · HTML: https://mlsanity.com/paper/2606.11628/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 68 of its 80 references that are in the map.
- First posted on arXiv 10 Jun 2026.

## Builds on (most cited first, 30 of 68)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [The "something something" video database for learning and evaluating visual common sense](https://mlsanity.com/paper/1706.04261/) (2017)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [Solving Rubik's Cube with a Robot Hand](https://mlsanity.com/paper/1910.07113/) (2019)
- [Rescaling Egocentric Vision](https://mlsanity.com/paper/2006.13256/) (2020)
- [World Action Models are Zero-shot Policies](https://mlsanity.com/paper/2602.15922/) (2026)
- [Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots](https://mlsanity.com/paper/2402.10329/) (2024)
- [Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers](https://mlsanity.com/paper/2402.19479/) (2024)
- [TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policies](https://mlsanity.com/paper/2412.10345/) (2024)
- [Any-point Trajectory Modeling for Policy Learning](https://mlsanity.com/paper/2401.00025/) (2023)
- [DexMV: Imitation Learning for Dexterous Manipulation from Human Videos](https://mlsanity.com/paper/2108.05877/) (2021)
- [EgoMimic: Scaling Imitation Learning via Egocentric Video](https://mlsanity.com/paper/2410.24221/) (2024)
- [ViPE: Video Pose Engine for 3D Geometric Perception](https://mlsanity.com/paper/2508.10934/) (2025)
- [DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation](https://mlsanity.com/paper/2403.07788/) (2024)
- [mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAs](https://mlsanity.com/paper/2512.15692/) (2025)
- [DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos](https://mlsanity.com/paper/2602.06949/) (2026)
- [Flow as the Cross-Domain Manipulation Interface](https://mlsanity.com/paper/2407.15208/) (2024)
- [EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos](https://mlsanity.com/paper/2507.12440/) (2025)
- [Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulation](https://mlsanity.com/paper/2405.01527/) (2024)
- [WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wild](https://mlsanity.com/paper/2409.12259/) (2024)
- [Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videos](https://mlsanity.com/paper/2507.15597/) (2025)
- [DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Video](https://mlsanity.com/paper/2202.00164/) (2022)
- [Dreamitate: Real-World Visuomotor Policy Learning via Video Generation](https://mlsanity.com/paper/2406.16862/) (2024)
- [General Flow as Foundation Affordance for Scalable Robot Learning](https://mlsanity.com/paper/2401.11439/) (2024)
- [PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation](https://mlsanity.com/paper/2601.03782/) (2026)
- [LEAP Hand: Low-Cost, Efficient, and Anthropomorphic Hand for Robot Learning](https://mlsanity.com/paper/2309.06440/) (2023)
- [Humanoid Policy ~ Human Policy](https://mlsanity.com/paper/2503.13441/) (2025)
- [Phantom: Training Robots Without Robots Using Only Human Videos](https://mlsanity.com/paper/2503.00779/) (2025)

## Built on by (most cited first, 5 of 5)

- [Act with Intent: Distilling Behavior Intent for Vision-Language-Action Models](https://mlsanity.com/paper/2608.23478/) (2026)
- [Track-and-Complete: Learning Humanoid Skills from a Single Failed Human Video](https://mlsanity.com/paper/2609.36924/) (2026)
- [GOTT: Object-centric Dexterous Manipulation with a Reusable Cross-Embodiment Primitive](https://mlsanity.com/paper/2610.03861/) (2026)
- [EigenDEXplore: Structured Exploration for Dexterous Manipulation with Human Priors](https://mlsanity.com/paper/2610.07681/) (2026)
- [DexCast: Learning Real-World Dexterous Control from Human Videos](https://mlsanity.com/paper/2610.14691/) (2026)

## Abstract (arXiv)

The most widely-adopted robot learning pipelines today learn skills from robot demonstrations or structured human data, which are expensive to collect and tied to specific embodiments. In contrast, unstructured human videos provide a scalable alternative. They contain diverse manipulation demonstrations across objects, scenes, and strategies, but are not directly connected to robot action. We propose LUCID, a two-stage framework that learns task intent from unstructured human videos drawn from internet-scale datasets and learns robot control in massively-parallel simulation. The intent model predicts short-horizon intent (what should happen next in the scene) from the current observation in closed loop. An embodiment-specific sensorimotor policy converts this intent into robot actions. The intent interface is shared across controllers, so the same intent model can be applied to different embodiments, from our primary dexterous hand to a parallel-jaw gripper. We evaluate LUCID on five real-world manipulation tasks: stirring, wiping, and binning supervised by only internet video, with zero-shot transfer to novel scenes and object instances; and push-T and cable routing supervised by 1 hr each of self-collected smartphone video. Project page: https://lucid-robot.github.io/.