mlsanityData to 14 Oct 2026

DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm System

arXiv 1910.03135 ↗First posted 7 Oct 2019cs.CV cs.LG cs.ROcited by 114

Not on the map. Why?

Abstract (arXiv)

Teleoperation offers the possibility of imparting robotic systems with sophisticated reasoning skills, intuition, and creativity to perform tasks. However, current teleoperation solutions for high degree-of-actuation (DoA), multi-fingered robots are generally cost-prohibitive, while low-cost offerings usually provide reduced degrees of control. Herein, a low-cost, vision based teleoperation system, DexPilot, was developed that allows for complete control over the full 23 DoA robotic system by merely observing the bare human hand. DexPilot enables operators to carry out a variety of complex manipulation tasks that go beyond simple pick-and-place operations. This allows for collection of high dimensional, multi-modality, state-action data that can be leveraged in the future to learn sensorimotor policies for challenging manipulation tasks. The system performance was measured through speed and reliability metrics across two human demonstrators on a variety of tasks. The videos of the experiments can be found at https://sites.google.com/view/dex-pilot.

Key facts. Cited by 114 papers in the map (data through 14 Oct 2026). First posted on arXiv 7 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 1; 2020: 1; 2021: 6; 2022: 19; 2023: 11; 2024: 25; 2025: 21; 2026: 30.

114 papers.

Sort:

By area:

  1. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  2. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 327
  3. 2021DexYCB: A Benchmark for Capturing Hand Grasping of Objectscited by 325
  4. 2022Masked Visual Pre-training for Motor Controlcited by 248
  5. 2021DexMV: Imitation Learning for Dexterous Manipulation from Human Videoscited by 206
  6. 2024OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learningcited by 189
  7. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 175
  8. 2024DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulationcited by 160
  9. 2023AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation Systemcited by 152
  10. 2024Open-TeleVision: Teleoperation with Immersive Active Visual Feedbackcited by 145
  11. 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 141
  12. 2022OakInk: A Large-scale Knowledge Repository for Understanding Hand-Object Interactioncited by 131
38 more
  1. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representationscited by 116
  2. 2022VideoDex: Learning Dexterity from Internet Videoscited by 106
  3. 2020State-Only Imitation Learning for Dexterous Manipulationcited by 91
  4. 2022DexVIP: Learning Dexterous Grasping with Human Hand Pose Priors from Videocited by 89
  5. 2025DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Graspingcited by 89
  6. 2022Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on Youtubecited by 82
  7. 2022From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperationcited by 82
  8. 2024OAKINK2: A Dataset of Bimanual Hands-Object Manipulation in Complex Task Completioncited by 82
  9. 2023LEAP Hand: Low-Cost, Efficient, and Anthropomorphic Hand for Robot Learningcited by 79
  10. 2022Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulationcited by 78
  11. 2024Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learningcited by 77
  12. 2025Scalable Vision-Language-Action Model Pretraining for Robotic Manipulation with Real-Life Human Activity Videoscited by 70
  13. 2023Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Playcited by 62
  14. 2022Holo-Dex: Teaching Dexterity with Immersive Mixed Realitycited by 57
  15. 2022Learning Generalizable Dexterous Manipulation from Human Grasp Affordancecited by 51
  16. 2024Twisting Lids Off with Two Handscited by 42
  17. 2023Dexterous Functional Graspingcited by 39
  18. 2025DexWild: Dexterous Human Interactions for In-the-Wild Robot Policiescited by 38
  19. 2024ViViDex: Learning Vision-based Dexterous Manipulation from Human Videoscited by 35
  20. 2022Learning to Imitate Object Interactions from Internet Videoscited by 34
  21. 2024ACE: A Cross-Platform Visual-Exoskeletons System for Low-Cost Dexterous Teleoperationcited by 34
  22. 2025BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activitiescited by 31
  23. 2024TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulationcited by 30
  24. 2022Learning Dexterous Manipulation from Exemplar Object Trajectories and Pre-Graspscited by 29
  25. 2024Bridging the Human to Robot Dexterity Gap through Object-Oriented Rewardscited by 28
  26. 2024Cross-Embodiment Dexterous Grasping with Reinforcement Learningcited by 27
  27. 2025In-N-On: Scaling Egocentric Manipulation with in-the-wild and on-task Datacited by 26
  28. 2024Bimanual Dexterity for Complex Taskscited by 25
  29. 2023DexDeform: Dexterous Deformable Object Manipulation with Human Demonstrations and Differentiable Physicscited by 23
  30. 2023GraspGF: Learning Score-based Grasping Primitive for Human-assisting Dexterous Graspingcited by 23
  31. 2022DexTransfer: Real World Multi-fingered Dexterous Grasping with Minimal Human Demonstrationscited by 22
  32. 2023Deformer: Dynamic Fusion Transformer for Robust Hand Pose Estimationcited by 21
  33. 2025Dexterous Manipulation through Imitation Learning: A Surveycited by 21
  34. 2024Single-to-Dual-View Adaptation for Egocentric 3D Hand Pose Estimationcited by 19
  35. 2021Shaping embodied agent behavior with activity-context priors from egocentric videocited by 17
  36. 2022Efficient Annotation and Learning for 3D Hand Pose Estimation: A Surveycited by 17
  37. 2022Tracking and Reconstructing Hand Object Interactions from Point Cloud Sequences in the Wildcited by 17
  38. 2023ContactArt: Learning 3D Interaction Priors for Category-level Articulated Object and Hand Poses Estimationcited by 16

50 most cited of 114 All 114 (JSON)