HandEdit: A Unified Benchmark for Egocentric Human-to-Robot Dexterous Hand Image Editing
Not on the map. Why?
Abstract (arXiv)
Robotic manipulation with dexterous hands is a cornerstone of Embodied AI, yet its progress is stifled by the high cost of collecting embodiment-aware teleoperation data. While abundant egocentric videos of human hands offer a scalable alternative, the profound discrepancies in appearance, articulation, and camera viewpoints between human and robotic data raise significant challenges for co-training. Though existing general image-editing models demonstrate strong capabilities, they lack necessary embodiment-specific priors to fully bridge this gap. In this work, we present HandEdit, a unified large-scale embodiment-aware image-editing dataset and benchmark specifically designed to transform human hands and arms into various dexterous robotic embodiments within egocentric frames. HandEdit comprises over 200M editing instances derived from five diverse source datasets, covering 26 distinct URDFs, including 13 hand-only and 13 hand-arm configurations. Alongside the dataset, we establish a unified benchmark protocol with two tracks: Hand-only and Hand-Arm, supporting URDF-conditioned evaluation. We conduct extensive evaluations of 11 representative image-editing baselines using a multi-dimensional metric suite, including generic similarity metrics, VLM-based judgment, and embodiment-aware metrics. HandEdit serves as a critical resource at the intersection of image editing and robotics: it advances embodiment-aware editing models while enabling scalable dexterous robotic learning from abundant human video data, paving the way for more generalizable Embodied AI.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 72 of its 85 references that are in the map. First posted on arXiv 12 Aug 2026.
Builds on
72 of 85 references.
References · 72
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
- 2024GPT-4o System Cardcited by 4,588
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
- 2025SAM 3: Segment Anything with Conceptscited by 906
- 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
38 more
- 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 657
- 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 647
- 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 624
- 2026World Action Models are Zero-shot Policiescited by 465
- 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 405
- 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 394
- 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 327
- 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 324
- 2022ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulationcited by 312
- 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 268
- 2021H2O: Two Hands Manipulating Objects for First Person Interaction Recognitioncited by 254
- 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 241
- 2022Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpaintingcited by 233
- 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 211
- 2023VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluationcited by 201
- 2023ProPainter: Improving Propagation and Transformer for Video Inpaintingcited by 197
- 2024HQ-Edit: A High-Quality Dataset for Instruction-based Image Editingcited by 193
- 2024EgoMimic: Scaling Imitation Learning via Egocentric Videocited by 175
- 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 175
- 2023Holistic Evaluation of Text-To-Image Modelscited by 160
- 2023AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation Systemcited by 152
- 2025HunyuanImage 3.0 Technical Reportcited by 149
- 2024OmniEdit: Building Image Editing Generalist Models Through Specialist Supervisioncited by 145
- 2026DreamDojo: A Generalist Robot World Model from Large-Scale Human Videoscited by 135
- 2024HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videoscited by 129
- 2024SEED-Data-Edit Technical Report: A Hybrid Dataset for Instructional Image Editingcited by 97
- 2026EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Datacited by 94
- 2025EgoLife: Towards Egocentric Life Assistantcited by 91
- 2022DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Videocited by 89
- 2024OAKINK2: A Dataset of Bimanual Hands-Object Manipulation in Complex Task Completioncited by 82
- 2024Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learningcited by 77
- 2024UMI on Legs: Making Manipulation Policies Mobile with Manipulation-Centric Whole-body Controllerscited by 75
- 2025Phantom: Training Robots Without Robots Using Only Human Videoscited by 70
- 2024OKAMI: Teaching Humanoid Robots Manipulation Skills through Single Video Imitationcited by 67
- 2023DexArt: Benchmarking Generalizable Dexterous Manipulation with Articulated Objectscited by 66
- 2025DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulationcited by 66
- 2022Harmonizer: Learning to Perform White-Box Image and Video Harmonizationcited by 61
- 2026EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the Worldcited by 57
50 most cited of 72 All 72 (JSON)
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026Bench2Dex: Benchmarking Visuo-Tactile Bimanual Dexterous Manipulation Across Dexterous Handscited by 2
- 2026Think Locally, Refine Globally for Memory-Efficient 3D Reconstructioncited by 1
- 2026MachEmbodied-U0: Unified Understanding and Generation Model for Embodied Intelligencecited by 1
- 2026Human-Centric Intelligence in the Era of Foundation Models: A Surveycited by 0
- 2026Skel-WAM: A Hand-Skeleton-Conditioned World Action Model for Human-to-Robot Manipulation Transfercited by 0
- 2026MachEmbodied-U0: Unified Understanding and Generation Model for Embodied Intelligence22 Sep 2026
- 2026Skel-WAM: A Hand-Skeleton-Conditioned World Action Model for Human-to-Robot Manipulation Transfer18 Sep 2026
- 2026Think Locally, Refine Globally for Memory-Efficient 3D Reconstruction18 Sep 2026
- 2026Bench2Dex: Benchmarking Visuo-Tactile Bimanual Dexterous Manipulation Across Dexterous Hands14 Sep 2026
- 2026Human-Centric Intelligence in the Era of Foundation Models: A Survey18 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2026IronMind: Scaling Humanoid Dexterous Manipulation via Camera-Space Ego-Centric Pretraining ↗
- 2026Wh0: Generative World Models as Scalable Sources of Egocentric Human Hand Manipulation Data
- 2026EgoJEPA: Beyond Retargeting for Dexterous Manipulation from Egocentric Videos
- 2026EgoMatrix: Physically Faithful Dexterous Robot Demonstrations from Egocentric Human Videos ↗
- 2026Open-AoE: An Open Egocentric Manipulation Dataset and Toolchain for Embodied Learning
- 2026H2R-Bench: Benchmarking Human-to-Robot Manipulation Video Generation in World Models
- 2026RoboReact: Agentic Skill Distillation from Generated Egocentric Videos for Generalizable Whole-Body Manipulation
- 2026EgoSteer: An Open-Source Full-Stack System Towards Steerable Dexterous Manipulation from Egocentric Videos