# MediaPipe Hands: On-device Real-time Hand Tracking

arXiv: https://arxiv.org/abs/2006.10214 · HTML: https://mlsanity.com/paper/2006.10214/

## Key facts

- Cited by 161 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2020.

## Built on by (most cited first, 30 of 161)

- [Effective Whole-body Pose Estimation with Two-stages Distillation](https://mlsanity.com/paper/2307.15880/) (2023)
- [Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Dataset](https://mlsanity.com/paper/2307.00818/) (2023)
- [AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation System](https://mlsanity.com/paper/2307.04577/) (2023)
- [One-Stage 3D Whole-Body Mesh Recovery with Component Aware Transformer](https://mlsanity.com/paper/2303.16160/) (2023)
- [WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wild](https://mlsanity.com/paper/2409.12259/) (2024)
- [PyTorch Geometric Temporal: Spatiotemporal Signal Processing with Neural Machine Learning Models](https://mlsanity.com/paper/2104.07788/) (2021)
- [An Outlook into the Future of Egocentric Vision](https://mlsanity.com/paper/2308.07123/) (2023)
- [Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications](https://mlsanity.com/paper/2510.07077/) (2025)
- [From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperation](https://mlsanity.com/paper/2204.12490/) (2022)
- [Dexterous Imitation Made Easy: A Learning-Based Framework for Efficient Dexterous Manipulation](https://mlsanity.com/paper/2203.13251/) (2022)
- [HOLD: Category-agnostic 3D Reconstruction of Interacting Hands and Objects from Video](https://mlsanity.com/paper/2311.18448/) (2023)
- [Holo-Dex: Teaching Dexterity with Immersive Mixed Reality](https://mlsanity.com/paper/2210.06463/) (2022)
- [CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Models](https://mlsanity.com/paper/2412.12093/) (2024)
- [HandRefiner: Refining Malformed Hands in Generated Images by Diffusion-based Conditional Inpainting](https://mlsanity.com/paper/2311.17957/) (2023)
- [HanDiffuser: Text-to-Image Generation With Realistic Hand Appearances](https://mlsanity.com/paper/2403.01693/) (2024)
- [ACE: A Cross-Platform Visual-Exoskeletons System for Low-Cost Dexterous Teleoperation](https://mlsanity.com/paper/2408.11805/) (2024)
- [HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interaction](https://mlsanity.com/paper/2406.06843/) (2024)
- [Self-Guidance: Boosting Flow and Diffusion Generation on Their Own](https://mlsanity.com/paper/2412.05827/) (2024)
- [EvHandPose: Event-based 3D Hand Pose Estimation with Sparse Supervision](https://mlsanity.com/paper/2303.02862/) (2023)
- [Complementing Event Streams and RGB Frames for Hand Mesh Reconstruction](https://mlsanity.com/paper/2403.07346/) (2024)
- [BlazePose GHUM Holistic: Real-time 3D Human Landmarks and Pose Estimation](https://mlsanity.com/paper/2206.11678/) (2022)
- [Authentic Hand Avatar from a Phone Scan via Universal Hand Model](https://mlsanity.com/paper/2405.07933/) (2024)
- [In My Perspective, In My Hands: Accurate Egocentric 2D Hand Pose and Action Recognition](https://mlsanity.com/paper/2404.09308/) (2024)
- [Hand1000: Generating Realistic Hands from Text with Only 1,000 Images](https://mlsanity.com/paper/2408.15461/) (2024)
- [AttentionHand: Text-driven Controllable Hand Image Generation for 3D Hand Reconstruction in the Wild](https://mlsanity.com/paper/2407.18034/) (2024)
- [RealisHuman: A Two-Stage Approach for Refining Malformed Human Parts in Generated Images](https://mlsanity.com/paper/2409.03644/) (2024)
- [Thermodynamics-informed neural networks for physically realistic mixed reality](https://mlsanity.com/paper/2210.13414/) (2022)
- [Blur2Blur: Blur Conversion for Unsupervised Image Deblurring on Unknown Domains](https://mlsanity.com/paper/2403.16205/) (2024)
- [HOIST-Former: Hand-held Objects Identification, Segmentation, and Tracking in the Wild](https://mlsanity.com/paper/2404.13819/) (2024)
- [RHanDS: Refining Malformed Hands for Generated Images with Decoupled Structure and Style Guidance](https://mlsanity.com/paper/2404.13984/) (2024)

## Abstract (arXiv)

We present a real-time on-device hand tracking pipeline that predicts hand skeleton from single RGB camera for AR/VR applications. The pipeline consists of two models: 1) a palm detector, 2) a hand landmark model. It's implemented via MediaPipe, a framework for building cross-platform ML solutions. The proposed model and pipeline architecture demonstrates real-time inference speed on mobile GPUs and high prediction quality. MediaPipe Hands is open sourced at https://mediapipe.dev.