mlsanityData to 14 Oct 2026

TeleOpBench: A Simulator-Centric Benchmark for Dual-Arm Dexterous Teleoperation

arXiv 2505.12748 ↗First posted 19 May 2025cs.RO cs.AI cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Teleoperation is a cornerstone of embodied-robot learning, and bimanual dexterous teleoperation in particular provides rich demonstrations that are difficult to obtain with fully autonomous systems. While recent studies have proposed diverse hardware pipelines-ranging from inertial motion-capture gloves to exoskeletons and vision-based interfaces-there is still no unified benchmark that enables fair, reproducible comparison of these systems. In this paper, we introduce TeleOpBench, a simulator-centric benchmark tailored to bimanual dexterous teleoperation. TeleOpBench contains 30 high-fidelity task environments that span pick-and-place, tool use, and collaborative manipulation, covering a broad spectrum of kinematic and force-interaction difficulty. Within this benchmark we implement four representative teleoperation modalities-(i) MoCap, (ii) VR device, (iii) arm-hand exoskeletons, and (iv) monocular vision tracking-and evaluate them with a common protocol and metric suite. To validate that performance in simulation is predictive of real-world behavior, we conduct mirrored experiments on a physical dual-arm platform equipped with two 6-DoF dexterous hands. Across 10 held-out tasks we observe a strong correlation between simulator and hardware performance, confirming the external validity of TeleOpBench. TeleOpBench establishes a common yardstick for teleoperation research and provides an extensible platform for future algorithmic and hardware innovation. Codes is now available at https://github.com/cyjdlhy/TeleOpBench .

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 50 of its 67 references that are in the map. First posted on arXiv 19 May 2025.

Builds on

50 of 67 references.

References · 50

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  3. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  5. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  6. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  7. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  8. 2018DeepMind Control Suitecited by 956
  9. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
  10. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  11. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
  12. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511
38 more
  1. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 391
  2. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 384
  3. 2020dm_control: Software and Tasks for Continuous Controlcited by 375
  4. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 260
  5. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Playcited by 223
  6. 2023Perpetual Humanoid Control for Real-time Simulated Avatarscited by 218
  7. 2023GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulatorscited by 189
  8. 2022Human-to-Robot Imitation in the Wildcited by 161
  9. 2023SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimationcited by 155
  10. 2023AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation Systemcited by 152
  11. 2023Structured World Models from Human Videoscited by 147
  12. 2024Open-TeleVision: Teleoperation with Immersive Active Visual Feedbackcited by 145
  13. 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 141
  14. 2023FurnitureBench: Reproducible Real-World Benchmark for Long-Horizon Complex Manipulationcited by 133
  15. 2019DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm Systemcited by 114
  16. 2024HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulationcited by 112
  17. 2022VideoDex: Learning Dexterity from Internet Videoscited by 106
  18. 2022From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperationcited by 82
  19. 2022MyoSuite -- A contact-rich simulation suite for musculoskeletal motor controlcited by 80
  20. 2024Learning Visuotactile Skills with Two Multifingered Handscited by 76
  21. 2025HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpitcited by 76
  22. 2024OPEN TEACH: A Versatile Teleoperation System for Robotic Manipulationcited by 71
  23. 2019IKEA Furniture Assembly Environment for Long-Horizon Complex Manipulation Taskscited by 69
  24. 2022Holo-Dex: Teaching Dexterity with Immersive Mixed Realitycited by 57
  25. 2023Deep Imitation Learning for Humanoid Loco-manipulation through Human Teleoperationcited by 54
  26. 2024Vision-based Manipulation from Single Human Video with Open-World Object Graphscited by 50
  27. 2023AirExo: Low-Cost Exoskeletons for Learning Whole-Arm Manipulation in the Wildcited by 47
  28. 2023LocoMuJoCo: A Comprehensive Imitation Learning Benchmark for Locomotioncited by 33
  29. 2024Adaptive Mobile Manipulation for Articulated Objects In the Open Worldcited by 31
  30. 2024CyberDemo: Augmenting Simulated Human Demonstration for Real-World Dexterous Manipulationcited by 28
  31. 2022RB2: Robotic Manipulation Benchmarking with a Twistcited by 23
  32. 2019Whole-Body Geometric Retargeting for Humanoid Robotscited by 20
  33. 2017Baxter's Homunculus: Virtual Reality Spaces for Teleoperation in Manufacturingcited by 19
  34. 2021NimbRo Avatar: Interactive Immersive Telepresence with Force-Feedback Telemanipulationcited by 13
  35. 2019Simitate: A Hybrid Imitation Learning Benchmarkcited by 10
  36. 2022Accelerating Interactive Human-like Manipulation Learning with GPU-based Simulation and High-quality Demonstrationscited by 8
  37. 2021GraspLook: a VR-based Telemanipulation System with R-CNN-driven Augmentation of Virtual Environmentcited by 7
  38. 20243D-Consistent Human Avatars with Sparse Inputs via Gaussian Splatting and Contrastive Learningcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.