mlsanityData to 14 Oct 2026

Open-TeleVision: Teleoperation with Immersive Active Visual Feedback

arXiv 2407.01512 ↗First posted 1 Jul 2024cs.RO cs.HC cs.LGcited by 145

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Teleoperation serves as a powerful method for collecting on-robot data essential for robot learning from demonstrations. The intuitiveness and ease of use of the teleoperation system are crucial for ensuring high-quality, diverse, and scalable data. To achieve this, we propose an immersive teleoperation system Open-TeleVision that allows operators to actively perceive the robot's surroundings in a stereoscopic manner. Additionally, the system mirrors the operator's arm and hand movements on the robot, creating an immersive experience as if the operator's mind is transmitted to a robot embodiment. We validate the effectiveness of our system by collecting data and training imitation learning policies on four long-horizon, precise tasks (Can Sorting, Can Insertion, Folding, and Unloading) for 2 different humanoid robots and deploy them in the real world. The system is open-sourced at: https://robot-tv.github.io/

Key facts. Cited by 145 papers in the map (data through 14 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 61 of its 82 references that are in the map. First posted on arXiv 1 Jul 2024.

Builds on

61 of 82 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024XBG: End-to-end Imitation Learning for Autonomous Behaviour in Human-Robot Interaction and Collaboration22 Jun
  2. 2024DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation12 Mar
  3. 2024CyberDemo: Augmenting Simulated Human Demonstration for Real-World Dexterous Manipulation22 Feb
  4. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation4 Jan
  5. 2023GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators22 Sep
  6. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware23 Apr
  7. 2023MimicPlay: Long-Horizon Imitation Learning by Watching Human Play24 Feb
  8. 2022From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperation26 Apr
  9. 2021The Surprising Effectiveness of Representation Learning for Visual Imitation2 Dec

Other references · 52

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  4. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  5. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  6. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
  7. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
  8. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
  9. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 551
  10. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 508
  11. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 394
  12. 2020Learning Agile Robotic Locomotion Skills by Imitating Animalscited by 251
38 more
  1. 2024OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learningcited by 189
  2. 2024HumanPlus: Humanoid Shadowing and Imitation from Humanscited by 185
  3. 2022Rapid Locomotion via Reinforcement Learningcited by 168
  4. 2023Extreme Parkour with Legged Robotscited by 164
  5. 2022Human-to-Robot Imitation in the Wildcited by 161
  6. 2023AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation Systemcited by 152
  7. 2022Legged Locomotion in Challenging Terrains using Egocentric Visioncited by 151
  8. 2023Robot Parkour Learningcited by 149
  9. 2023Structured World Models from Human Videoscited by 147
  10. 2024Expressive Whole-Body Control for Humanoid Robotscited by 139
  11. 2024Learning Human-to-Humanoid Real-Time Whole-Body Teleoperationcited by 130
  12. 2024Yell At Your Robot: Improving On-the-Fly from Language Correctionscited by 124
  13. 2022Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotioncited by 121
  14. 2021Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robotscited by 116
  15. 2019DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm Systemcited by 114
  16. 2022VideoDex: Learning Dexterity from Internet Videoscited by 106
  17. 2023Synthesizing Physical Character-Scene Interactionscited by 104
  18. 2022From Play to Policy: Conditional Behavior Generation from Uncurated Robot Datacited by 103
  19. 2023On Bringing Robots Homecited by 101
  20. 2023CALM: Conditional Adversarial Latent Models for Directable Virtual Characterscited by 94
  21. 2021Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learningcited by 92
  22. 2022Adversarial Motion Priors Make Good Substitutes for Complex Reward Functionscited by 89
  23. 2022Robotic Telekinesis: Learning a Robotic Hand Imitator by Watching Humans on Youtubecited by 82
  24. 2021Minimizing Energy Consumption Leads to the Emergence of Gaits in Legged Robotscited by 76
  25. 2024Learning Visuotactile Skills with Two Multifingered Handscited by 76
  26. 2018Robot Learning in Homes: Improving Generalization and Reducing Dataset Biascited by 75
  27. 2022Holo-Dex: Teaching Dexterity with Immersive Mixed Realitycited by 57
  28. 2023Deep Imitation Learning for Humanoid Loco-manipulation through Human Teleoperationcited by 54
  29. 2024Vision-based Manipulation from Single Human Video with Open-World Object Graphscited by 50
  30. 2023AirExo: Low-Cost Exoskeletons for Learning Whole-Arm Manipulation in the Wildcited by 47
  31. 2023Learning Vision-Based Bipedal Locomotion for Challenging Terraincited by 42
  32. 2020UniCon: Universal Neural Controller For Physics-based Character Motioncited by 40
  33. 2023Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotioncited by 39
  34. 2022OPT-Mimic: Imitation of Optimized Trajectories for Dynamic Quadruped Behaviorscited by 35
  35. 2024Adaptive Mobile Manipulation for Articulated Objects In the Open Worldcited by 31
  36. 2023ALAN: Autonomously Exploring Robotic Agents in the Real Worldcited by 24
  37. 2023Generalized Animal Imitator: Agile Locomotion with Versatile Motion Priorcited by 19
  38. 2020A Mobile Robot Hand-Arm Teleoperation System by Vision and IMUcited by 15

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 25; 2025: 73; 2026: 47.

145 papers.

Sort:

By area:

  1. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 665
  2. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 229
  3. 2024EgoMimic: Scaling Imitation Learning via Egocentric Videocited by 175
  4. 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 175
  5. 2024Data Scaling Laws in Imitation Learning for Robotic Manipulationcited by 145
  6. 2025EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videoscited by 128
  7. 2025RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twinscited by 121
  8. 2024DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learningcited by 116
  9. 2025TWIST: Teleoperated Whole-Body Imitation Systemcited by 101
  10. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 84
  11. 2025Humanoid Policy ~ Human Policycited by 78
  12. 2025HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpitcited by 76
38 more
  1. 2025InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactionscited by 72
  2. 2024RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version)cited by 69
  3. 2024OKAMI: Teaching Humanoid Robots Manipulation Skills through Single Video Imitationcited by 67
  4. 2025ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learningcited by 66
  5. 2026EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the Worldcited by 57
  6. 2025RoboVerse: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learningcited by 54
  7. 2024Generalizable Humanoid Manipulation with 3D Diffusion Policiescited by 52
  8. 2024Mobile-TeleVision: Predictive Motion Priors for Humanoid Whole-Body Controlcited by 51
  9. 2025AMO: Adaptive Motion Optimization for Hyper-Dexterous Humanoid Whole-Body Controlcited by 51
  10. 2025ManiFlow: A General Robot Manipulation Policy via Consistency Flow Trainingcited by 49
  11. 2025Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoidscited by 48
  12. 2025Learning Humanoid Standing-up Control across Diverse Posturescited by 46
  13. 2025Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulationcited by 46
  14. 2024Robot Utility Models: General Policies for Zero-Shot Deployment in New Environmentscited by 44
  15. 2025TWIST2: Scalable, Portable, and Holistic Humanoid Data Collection Systemcited by 44
  16. 2025DexWild: Dexterous Human Interactions for In-the-Wild Robot Policiescited by 38
  17. 2025DexMachina: Functional Retargeting for Bimanual Dexterous Manipulationcited by 37
  18. 2024ForceMimic: Force-Centric Imitation Learning with Force-Motion Capture System for Contact-Rich Manipulationcited by 36
  19. 2025WholeBodyVLA: Towards Unified Latent VLA for Whole-Body Loco-Manipulation Controlcited by 36
  20. 2024ACE: A Cross-Platform Visual-Exoskeletons System for Low-Cost Dexterous Teleoperationcited by 34
  21. 2024ARCap: Collecting High-quality Human Demonstrations for Robot Learning with Augmented Reality Feedbackcited by 31
  22. 2025BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activitiescited by 31
  23. 2024Learning Smooth Humanoid Locomotion through Lipschitz-Constrained Policiescited by 30
  24. 2026RDT2: Exploring the Scaling Limit of UMI Data Towards Zero-Shot Cross-Embodiment Generalizationcited by 30
  25. 2025In-N-On: Scaling Egocentric Manipulation with in-the-wild and on-task Datacited by 26
  26. 2025Eye, Robot: Learning to Look to Act with a BC-RL Perception-Action Loopcited by 25
  27. 2025Humanoid Everyday: A Comprehensive Robotic Dataset for Open-World Humanoid Manipulationcited by 25
  28. 2025AC-DiT: Adaptive Coordination Diffusion Transformer for Mobile Manipulationcited by 23
  29. 2024TidyBot++: An Open-Source Holonomic Mobile Manipulator for Robot Learningcited by 22
  30. 2025Being-0: A Humanoid Robotic Agent with Vision-Language Models and Modular Skillscited by 22
  31. 2025Dexterous Manipulation through Imitation Learning: A Surveycited by 21
  32. 2025Dexonomy: Synthesizing All Dexterous Grasp Types in a Grasp Taxonomycited by 20
  33. 2025DEXOP: A Device for Robotic Transfer of Dexterous Human Manipulationcited by 19
  34. 2025ActiveUMI: Robotic Manipulation with Active Perception from Robot-Free Human Demonstrationscited by 19
  35. 2025GSWorld: Closed-Loop Photo-Realistic Simulation Suite for Robotic Manipulationcited by 19
  36. 2024SOLAMI: Social Vision-Language-Action Modeling for Immersive Interaction with 3D Autonomous Characterscited by 18
  37. 2025SAIL: Faster-than-Demonstration Execution of Imitation Learning Policiescited by 18
  38. 2024AnyBimanual: Transferring Unimanual Policy for General Bimanual Manipulationcited by 16

50 most cited of 145 All 145 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Active Vision Might Be All You Need: Exploring Active Vision in Bimanual Robotic Manipulation
  2. 2024TelePreview: A User-Friendly Teleoperation System with Virtual Arm Assistance for Enhanced Effectiveness ↗
  3. 2025Open TeleDex: A Hardware-Agnostic Teleoperation System for Imitation Learning based Dexterous Manipulation
  4. 2025Echo: An Open-Source, Low-Cost Teleoperation System with Force Feedback for Dataset Collection in Robot Learning
  5. 2024Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learning
  6. 2025Learning Adaptive Neural Teleoperation for Humanoid Robots: From Inverse Kinematics to End-to-End Control ↗
  7. 2026RynnWorld-Teleop: An Action-Conditioned World Model for Digital Teleoperation
  8. 2024TeleMoMa: A Modular and Versatile Teleoperation System for Mobile Manipulation