mlsanityData to 7 Oct 2026

Single RGB-D Camera Teleoperation for General Robotic Manipulation

arXiv 2106.14396 ↗First posted 28 Jun 2021cs.RO cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

We propose a teleoperation system that uses a single RGB-D camera as the human motion capture device. Our system can perform general manipulation tasks such as cloth folding, hammering and 3mm clearance peg in hole. We propose the use of non-Cartesian oblique coordinate frame, dynamic motion scaling and reposition of operator frames to increase the flexibility of our teleoperation system. We hypothesize that lowering the barrier of entry to teleoperation will allow for wider deployment of supervised autonomy system, which will in turn generates realistic datasets that unlock the potential of machine learning for robotic manipulation. Demo of our systems are available online https://sites.google.com/view/manipulation-teleop-with-rgbd

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 26 of its 60 references that are in the map. First posted on arXiv 28 Jun 2021.

Builds on

26 of 60 references.

References · 26

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  3. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  4. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 268
  5. 2018Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experiencecited by 203
  6. 2018RoboTurk: A Crowdsourcing Platform for Robotic Skill Learning through Imitationcited by 202
  7. 20193D Hand Shape and Pose from Images in the Wildcited by 190
  8. 2020MediaPipe Hands: On-device Real-time Hand Trackingcited by 157
  9. 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 140
  10. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  11. 2020Visual Imitation Made Easycited by 111
  12. 2019DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm Systemcited by 105
14 more
  1. 2019Grasping in the Wild:Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrationscited by 104
  2. 2020Human-in-the-Loop Imitation Learning using Remote Teleoperationcited by 103
  3. 2018Multiple Interactions Made Easy (MIME): Large Scale Demonstrations Data for Imitationcited by 76
  4. 2020FrankMocap: Fast Monocular 3D Hand and Body Motion Capture by Regression and Integrationcited by 72
  5. 2018Learning Task-Oriented Grasping for Tool Manipulation from Simulated Self-Supervisioncited by 50
  6. 2021Auto-Tuned Sim-to-Real Transfercited by 45
  7. 2020Learning Multi-Arm Manipulation Through Collaborative Teleoperationcited by 31
  8. 2016More than a Million Ways to Be Pushed: A High-Fidelity Experimental Dataset of Planar Pushingcited by 30
  9. 2019DoorGym: A Scalable Door Opening Environment And Baseline Agentcited by 26
  10. 2018Task-Oriented Hand Motion Retargeting for Dexterous Manipulation Imitationcited by 25
  11. 2019How to pick the domain randomization parameters for sim-to-real transfer of reinforcement learning policies?cited by 25
  12. 2018Vision-based Teleoperation of Shadow Dexterous Hand using End-to-End Deep Neural Networkcited by 19
  13. 2020A Mobile Robot Hand-Arm Teleoperation System by Vision and IMUcited by 15
  14. 2019Multi-task Batch Reinforcement Learning with Metric Learningcited by 1

Built on by

Cited by year

Citing papers per year: 2023: 1.

1 paper.

  1. 2023Chain-of-Thought Predictive Controlcited by 18