mlsanityData to 7 Oct 2026

AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System

arXiv 2307.04577 ↗First posted 10 Jul 2023cs.RO cs.CV cs.LGcited by 136

Not on the map. Why?

Cited by 23 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Vision-based teleoperation offers the possibility to endow robots with human-level intelligence to physically interact with the environment, while only requiring low-cost camera sensors. However, current vision-based teleoperation systems are designed and engineered towards a particular robot model and deploy environment, which scales poorly as the pool of the robot models expands and the variety of the operating environment increases. In this paper, we propose AnyTeleop, a unified and general teleoperation system to support multiple different arms, hands, realities, and camera configurations within a single system. Although being designed to provide great flexibility to the choice of simulators and real hardware, our system can still achieve great performance. For real-world experiments, AnyTeleop can outperform a previous system that was designed for a specific robot hardware with a higher success rate, using the same robot. For teleoperation in simulation, AnyTeleop leads to better imitation learning performance, compared with a previous system that is particularly designed for that simulator. Project page: https://yzqin.github.io/anyteleop/.

Key facts. Cited by 136 papers in the map (data through 7 Oct 2026). Builds on 36 of its 66 references that are in the map. First posted on arXiv 10 Jul 2023.

Builds on

36 of 66 references.

References · 36

  1. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  2. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  3. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  4. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  5. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  6. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  7. 2020ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulationcited by 249
  8. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 241
  9. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitationcited by 202
  10. 2021DexMV: Imitation Learning for Dexterous Manipulation from Human Videoscited by 194
  11. 2020MediaPipe Hands: On-device Real-time Hand Trackingcited by 157
  12. 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 140
24 more
  1. 2019DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm Systemcited by 105
  2. 2020Human-in-the-Loop Imitation Learning using Remote Teleoperationcited by 103
  3. 2022VideoDex: Learning Dexterity from Internet Videoscited by 101
  4. 2022Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on Youtubecited by 79
  5. 2022From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperationcited by 79
  6. 2022DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulationcited by 78
  7. 2022Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulationcited by 75
  8. 2020FrankMocap: Fast Monocular 3D Hand and Body Motion Capture by Regression and Integrationcited by 72
  9. 2022Holo-Dex: Teaching Dexterity with Immersive Mixed Realitycited by 52
  10. 2018Riemannian Motion Policiescited by 45
  11. 2022Learning Continuous Grasping Function with a Dexterous Hand from Human Demonstrationscited by 34
  12. 2020Learning Multi-Arm Manipulation Through Collaborative Teleoperationcited by 31
  13. 2018Task-Oriented Hand Motion Retargeting for Dexterous Manipulation Imitationcited by 25
  14. 2017Baxter's Homunculus: Virtual Reality Spaces for Teleoperation in Manufacturingcited by 19
  15. 2018Vision-based Teleoperation of Shadow Dexterous Hand using End-to-End Deep Neural Networkcited by 19
  16. 2018RMPflow: A Computational Graph for Automatic Motion Policy Generationcited by 18
  17. 2020A Mobile Robot Hand-Arm Teleoperation System by Vision and IMUcited by 15
  18. 2020In-Hand Object Pose Tracking via Contact Feedback and GPU-Accelerated Robotic Simulationcited by 9
  19. 2022ASHA: Assistive Teleoperation via Human-in-the-Loop Reinforcement Learningcited by 8
  20. 2022Accelerating Interactive Human-like Manipulation Learning with GPU-based Simulation and High-quality Demonstrationscited by 8
  21. 2021GraspLook: a VR-based Telemanipulation System with R-CNN-driven Augmentation of Virtual Environmentcited by 7
  22. 2015Autonomy Infused Teleoperation with Application to BCI Manipulationcited by 6
  23. 2019Teleoperator Imitation with Continuous-time Safetycited by 5
  24. 2019A Continuous Teleoperation Subspace with Empirical and Algorithmic Mapping Algorithms for Non-Anthropomorphic Handscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 27; 2025: 55; 2026: 52.

136 papers.

Sort:

By area:

  1. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 216
  2. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 166
  3. 2024Open-TeleVision: Teleoperation with Immersive Active Visual Feedbackcited by 137
  4. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representationscited by 115
  5. 2025Reactive Diffusion Policy: Slow-Fast Visual-Tactile Policy Learning for Contact-Rich Manipulationcited by 103
  6. 2025Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videoscited by 94
  7. 2025DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Graspingcited by 87
  8. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  9. 2024Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learningcited by 69
  10. 20243D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensingcited by 67
  11. 2025Large Video Planner Enables Generalizable Robot Controlcited by 67
  12. 2025HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpitcited by 65
38 more
  1. 2025Scalable Vision-Language-Action Model Pretraining for Robotic Manipulation with Real-Life Human Activity Videoscited by 63
  2. 2024OKAMI: Teaching Humanoid Robots Manipulation Skills through Single Video Imitationcited by 62
  3. 2025ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learningcited by 56
  4. 2024RAM: Retrieval-Based Affordance Transfer for Generalizable Zero-Shot Robotic Manipulationcited by 55
  5. 2023Robot Synesthesia: In-Hand Manipulation with Visuotactile Sensingcited by 51
  6. 2024Generalizable Humanoid Manipulation with 3D Diffusion Policiescited by 49
  7. 2025AMO: Adaptive Motion Optimization for Hyper-Dexterous Humanoid Whole-Body Controlcited by 47
  8. 2024Twisting Lids Off with Two Handscited by 36
  9. 2024ViViDex: Learning Vision-based Dexterous Manipulation from Human Videoscited by 33
  10. 2024ACE: A Cross-Platform Visual-Exoskeletons System for Low-Cost Dexterous Teleoperationcited by 32
  11. 2024ARCap: Collecting High-quality Human Demonstrations for Robot Learning with Augmented Reality Feedbackcited by 31
  12. 2025DexMachina: Functional Retargeting for Bimanual Dexterous Manipulationcited by 30
  13. 2026RoboClaw: An Agentic Framework for Scalable Long-Horizon Robotic Taskscited by 30
  14. 2025BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activitiescited by 29
  15. 2024TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulationcited by 27
  16. 2024CyberDemo: Augmenting Simulated Human Demonstration for Real-World Dexterous Manipulationcited by 25
  17. 2025DexGrasp Anything: Towards Universal Robotic Dexterous Grasping with Physics Awarenesscited by 25
  18. 2025SPIDER: Scalable Physics-Informed Dexterous Retargetingcited by 25
  19. 2024Cross-Embodiment Dexterous Grasping with Reinforcement Learningcited by 24
  20. 2025Eva-VLA: Evaluating Vision-Language-Action Models' Robustness Under Real-World Physical Variationscited by 24
  21. 2024Ag2Manip: Learning Novel Manipulation Skills with Agent-Agnostic Visual and Action Representationscited by 23
  22. 2024Lessons from Learning to Spin "Pens"cited by 22
  23. 2024Bimanual Dexterity for Complex Taskscited by 22
  24. 2025RoboCopilot: Human-in-the-loop Interactive Imitation Learning for Robot Manipulationcited by 22
  25. 2025Humanoid Everyday: A Comprehensive Robotic Dataset for Open-World Humanoid Manipulationcited by 22
  26. 2025In-N-On: Scaling Egocentric Manipulation with in-the-wild and on-task Datacited by 21
  27. 2024GarmentLab: A Unified Simulation and Benchmark for Garment Manipulationcited by 19
  28. 2025Dexonomy: Synthesizing All Dexterous Grasp Types in a Grasp Taxonomycited by 19
  29. 2025DexVLG: Dexterous Vision-Language-Grasp Model at Scalecited by 19
  30. 2025Viser: Imperative, Web-based 3D Visualization in Pythoncited by 19
  31. 2025Dexterous Manipulation through Imitation Learning: A Surveycited by 18
  32. 2025CordViP: Correspondence-based Visuomotor Policy for Dexterous Manipulation in Real-Worldcited by 16
  33. 2024Learning Diverse Bimanual Dexterous Manipulation Skills from Human Demonstrationscited by 12
  34. 2025Geometric Retargeting: A Principled, Ultrafast Neural Hand Retargeting Algorithmcited by 12
  35. 2025How to Train Your Robots? The Impact of Demonstration Modality on Imitation Learningcited by 11
  36. 2024QuasiSim: Parameterized Quasi-Physical Simulators for Dexterous Manipulations Transfercited by 10
  37. 2024Human-Agent Joint Learning for Efficient Robot Manipulation Skill Acquisitioncited by 10
  38. 2025Learning to Transfer Human Hand Skills for Robot Manipulationscited by 10

50 most cited of 136 All 136 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024TelePreview: A User-Friendly Teleoperation System with Virtual Arm Assistance for Enhanced Effectiveness ↗
  2. 2025TypeTele: Releasing Dexterity in Teleoperation by Dexterous Manipulation Types
  3. 2025RAPID Hand: A Robust, Affordable, Perception-Integrated, Dexterous Manipulation Platform for Generalist Robot Autonomy
  4. 2025Open TeleDex: A Hardware-Agnostic Teleoperation System for Imitation Learning based Dexterous Manipulation
  5. 2025Dexterous Teleoperation of 20-DoF ByteDexter Hand via Human Motion Retargeting
  6. 2026AnyDexRT: Calibration-Free Dexterous Hand Retargeting with Few-Shot Human Guidance
  7. 2024Tilde: Teleoperation for Dexterous In-Hand Manipulation Learning with a DeltaHand
  8. 2026DexTele: A Dual-Arm Dexterous Teleoperation System Based on Motion Retargeting and Adaptive Force Control ↗