# ACE: A Cross-Platform Visual-Exoskeletons System for Low-Cost Dexterous Teleoperation

arXiv: https://arxiv.org/abs/2408.11805 · HTML: https://mlsanity.com/paper/2408.11805/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 51 references that are in the map.
- First posted on arXiv 21 Aug 2024.

## Builds on (most cited first, 30 of 32)

- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [Open X-Embodiment: Robotic Learning Datasets and RT-X Models](https://mlsanity.com/paper/2310.08864/) (2023)
- [BridgeData V2: A Dataset for Robot Learning at Scale](https://mlsanity.com/paper/2308.12952/) (2023)
- [BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning](https://mlsanity.com/paper/2202.02005/) (2022)
- [GRAB: A Dataset of Whole-Body Human Grasping of Objects](https://mlsanity.com/paper/2008.11200/) (2020)
- [Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots](https://mlsanity.com/paper/2402.10329/) (2024)
- [Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation](https://mlsanity.com/paper/2401.02117/) (2024)
- [ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulation](https://mlsanity.com/paper/2204.13662/) (2022)
- [Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation](https://mlsanity.com/paper/1710.04615/) (2017)
- [GELLO: A General, Low-Cost, and Intuitive Teleoperation Framework for Robot Manipulators](https://mlsanity.com/paper/2309.13037/) (2023)
- [MediaPipe Hands: On-device Real-time Hand Tracking](https://mlsanity.com/paper/2006.10214/) (2020)
- [Human-to-Robot Imitation in the Wild](https://mlsanity.com/paper/2207.09450/) (2022)
- [DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation](https://mlsanity.com/paper/2403.07788/) (2024)
- [AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System](https://mlsanity.com/paper/2307.04577/) (2023)
- [Open-TeleVision: Teleoperation with Immersive Active Visual Feedback](https://mlsanity.com/paper/2407.01512/) (2024)
- [DexPilot: Vision Based Teleoperation of Dexterous Robotic Hand-Arm System](https://mlsanity.com/paper/1910.03135/) (2019)
- [VideoDex: Learning Dexterity from Internet Videos](https://mlsanity.com/paper/2212.04498/) (2022)
- [From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperation](https://mlsanity.com/paper/2204.12490/) (2022)
- [Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulation](https://mlsanity.com/paper/2203.13251/) (2022)
- [FrankMocap: Fast Monocular 3D Hand and Body Motion Capture by Regression and Integration](https://mlsanity.com/paper/2008.08324/) (2020)
- [Error-Aware Imitation Learning from Teleoperation Data for Mobile Manipulation](https://mlsanity.com/paper/2112.05251/) (2021)
- [Holo-Dex: Teaching Dexterity with Immersive Mixed Reality](https://mlsanity.com/paper/2210.06463/) (2022)
- [Towards Generalizable Zero-Shot Manipulation via Translating Human Interaction Plans](https://mlsanity.com/paper/2312.00775/) (2023)
- [AirExo: Low-Cost Exoskeletons for Learning Whole-Arm Manipulation in the Wild](https://mlsanity.com/paper/2309.14975/) (2023)
- [Learning Continuous Grasping Function with a Dexterous Hand from Human Demonstrations](https://mlsanity.com/paper/2207.05053/) (2022)
- [Baxter's Homunculus: Virtual Reality Spaces for Teleoperation in Manufacturing](https://mlsanity.com/paper/1703.01270/) (2017)
- [Vision-based Teleoperation of Shadow Dexterous Hand using End-to-End Deep Neural Network](https://mlsanity.com/paper/1809.06268/) (2018)
- [A Mobile Robot Hand-Arm Teleoperation System by Vision and IMU](https://mlsanity.com/paper/2003.05212/) (2020)

## Built on by (most cited first, 30 of 34)

- [EgoMimic: Scaling Imitation Learning via Egocentric Video](https://mlsanity.com/paper/2410.24221/) (2024)
- [Humanoid Policy ~ Human Policy](https://mlsanity.com/paper/2503.13441/) (2025)
- [HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpit](https://mlsanity.com/paper/2502.13013/) (2025)
- [ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning](https://mlsanity.com/paper/2503.21860/) (2025)
- [Generalizable Humanoid Manipulation with 3D Diffusion Policies](https://mlsanity.com/paper/2410.10803/) (2024)
- [Robot Utility Models: General Policies for Zero-Shot Deployment in New Environments](https://mlsanity.com/paper/2409.05865/) (2024)
- [ForceMimic: Force-Centric Imitation Learning with Force-Motion Capture System for Contact-Rich Manipulation](https://mlsanity.com/paper/2410.07554/) (2024)
- [ARCap: Collecting High-quality Human Demonstrations for Robot Learning with Augmented Reality Feedback](https://mlsanity.com/paper/2410.08464/) (2024)
- [BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activities](https://mlsanity.com/paper/2503.05652/) (2025)
- [Bridging the Human to Robot Dexterity Gap through Object-Oriented Rewards](https://mlsanity.com/paper/2410.23289/) (2024)
- [Being-0: A Humanoid Robotic Agent with Vision-Language Models and Modular Skills](https://mlsanity.com/paper/2503.12533/) (2025)
- [Dexterous Manipulation through Imitation Learning: A Survey](https://mlsanity.com/paper/2504.03515/) (2025)
- [Dexonomy: Synthesizing All Dexterous Grasp Types in a Grasp Taxonomy](https://mlsanity.com/paper/2504.18829/) (2025)
- [DEXOP: A Device for Robotic Transfer of Dexterous Human Manipulation](https://mlsanity.com/paper/2509.04441/) (2025)
- [Dexterity from Smart Lenses: Multi-Fingered Robot Manipulation with In-the-Wild Human Demonstrations](https://mlsanity.com/paper/2511.16661/) (2025)
- [AnyBimanual: Transferring Unimanual Policy for General Bimanual Manipulation](https://mlsanity.com/paper/2412.06779/) (2024)
- [TextOp: Real-time Interactive Text-Driven Humanoid Robot Motion Generation and Control](https://mlsanity.com/paper/2602.07439/) (2026)
- [Learning to Transfer Human Hand Skills for Robot Manipulations](https://mlsanity.com/paper/2501.04169/) (2025)
- [ManiGaussian++: General Robotic Bimanual Manipulation with Hierarchical Gaussian World Model](https://mlsanity.com/paper/2506.19842/) (2025)
- [Human2LocoMan: Learning Versatile Quadrupedal Manipulation with Human Pretraining](https://mlsanity.com/paper/2506.16475/) (2025)
- [WildLMa: Long Horizon Loco-Manipulation in the Wild](https://mlsanity.com/paper/2411.15131/) (2024)
- [DexHiL: A Human-in-the-Loop Framework for Vision-Language-Action Model Post-Training in Dexterous Manipulation](https://mlsanity.com/paper/2603.09121/) (2026)
- [Real-Time Operator Takeover for Visuomotor Diffusion Policy Training](https://mlsanity.com/paper/2502.02308/) (2025)
- [Proleptic Temporal Ensemble for Improving the Speed of Robot Tasks Generated by Imitation Learning](https://mlsanity.com/paper/2410.16981/) (2024)
- [CLOT: Closed-Loop Global Motion Tracking for Whole-Body Humanoid Teleoperation](https://mlsanity.com/paper/2602.15060/) (2026)
- [Universal Manipulation Exoskeleton: Learning Compliant Whole-body Policies with Real-time Torque Feedback](https://mlsanity.com/paper/2606.14218/) (2026)
- [AhaRobot: A Low-Cost Open-Source Bimanual Mobile Manipulator for Embodied AI](https://mlsanity.com/paper/2503.10070/) (2025)
- [Human-Robot Copilot for Data-Efficient Imitation Learning](https://mlsanity.com/paper/2604.03613/) (2026)
- [Sim-and-Human Co-training for Data-Efficient and Scene-Generalizable Bimanual Manipulation](https://mlsanity.com/paper/2601.19406/) (2026)
- [Structural Action Transformer for 3D Dexterous Manipulation](https://mlsanity.com/paper/2603.03960/) (2026)

## Abstract (arXiv)

Learning from demonstrations has shown to be an effective approach to robotic manipulation, especially with the recently collected large-scale robot data with teleoperation systems. Building an efficient teleoperation system across diverse robot platforms has become more crucial than ever. However, there is a notable lack of cost-effective and user-friendly teleoperation systems for different end-effectors, e.g., anthropomorphic robot hands and grippers, that can operate across multiple platforms. To address this issue, we develop ACE, a cross-platform visual-exoskeleton system for low-cost dexterous teleoperation. Our system utilizes a hand-facing camera to capture 3D hand poses and an exoskeleton mounted on a portable base, enabling accurate real-time capture of both finger and wrist poses. Compared to previous systems, which often require hardware customization according to different robots, our single system can generalize to humanoid hands, arm-hands, arm-gripper, and quadruped-gripper systems with high-precision teleoperation. This enables imitation learning for complex manipulation tasks on diverse platforms.