mlsanityData to 7 Oct 2026

WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wild

arXiv 2409.12259 ↗First posted 18 Sep 2024cs.CVcited by 117

Not on the map. Why?

Abstract (arXiv)

In recent years, 3D hand pose estimation methods have garnered significant attention due to their extensive applications in human-computer interaction, virtual reality, and robotics. In contrast, there has been a notable gap in hand detection pipelines, posing significant challenges in constructing effective real-world multi-hand reconstruction systems. In this work, we present a data-driven pipeline for efficient multi-hand reconstruction in the wild. The proposed pipeline is composed of two components: a real-time fully convolutional hand localization and a high-fidelity transformer-based 3D hand reconstruction model. To tackle the limitations of previous methods and build a robust and stable detection network, we introduce a large-scale dataset with over than 2M in-the-wild hand images with diverse lighting, illumination, and occlusion conditions. Our approach outperforms previous methods in both efficiency and accuracy on popular 2D and 3D benchmarks. Finally, we showcase the effectiveness of our pipeline to achieve smooth 3D hand tracking from monocular videos, without utilizing any temporal components. Code, models, and dataset are available https://rolpotamias.github.io/WiLoR.

Key facts. Cited by 117 papers in the map (data through 7 Oct 2026). Builds on 80 of its 110 references that are in the map. First posted on arXiv 18 Sep 2024.

Builds on

80 of 110 references.

References · 80

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2017Mask R-CNNcited by 7,259
  5. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  6. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  7. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  8. 2018YOLOv3: An Incremental Improvementcited by 2,652
  9. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  10. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,417
  11. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  12. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
38 more
  1. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  2. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 913
  3. 2017End-to-end Recovery of Human Shape and Posecited by 883
  4. 2018Path Aggregation Network for Instance Segmentationcited by 796
  5. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 723
  6. 2019CenterNet: Keypoint Triplets for Object Detectioncited by 654
  7. 2022ViTPose: Simple Vision Transformer Baselines for Human Pose Estimationcited by 450
  8. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 417
  9. 2020Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detectioncited by 408
  10. 2019Learning joint reconstruction of hands and manipulated objectscited by 377
  11. 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 357
  12. 2023Humans in 4D: Reconstructing and Tracking Humans with Transformerscited by 344
  13. 2021DexYCB: A Benchmark for Capturing Hand Grasping of Objectscited by 320
  14. 2022ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulationcited by 306
  15. 2023Reconstructing Hands in 3D with Transformerscited by 298
  16. 2020Understanding Human Hands in Contact at Internet Scalecited by 281
  17. 2020I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Imagecited by 274
  18. 2019FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Imagescited by 271
  19. 2021Mesh Graphormercited by 264
  20. 2020Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Posecited by 248
  21. 2020InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Imagecited by 244
  22. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  23. 2017Hand Keypoint Detection in Single Images using Multiview Bootstrappingcited by 231
  24. 2023BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motioncited by 229
  25. 2020Towards Fast, Accurate and Stable 3D Dense Face Alignmentcited by 219
  26. 2023WHAM: Reconstructing World-grounded Humans with Accurate 3D Motioncited by 200
  27. 2022AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Timecited by 199
  28. 20193D Hand Shape and Pose from Images in the Wildcited by 190
  29. 2020Whole-Body Human Pose Estimation in the Wildcited by 178
  30. 2016Panoptic Studio: A Massively Multiview System for Social Interaction Capturecited by 173
  31. 2018Monocular Total Capture: Posing Face, Body, and Hands in the Wildcited by 162
  32. 2023FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterizationcited by 161
  33. 2020MediaPipe Hands: On-device Real-time Hand Trackingcited by 157
  34. 2023SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimationcited by 154
  35. 2022PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Imagescited by 152
  36. 2020Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wildcited by 146
  37. 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 140
  38. 2017Towards Accurate Multi-person Pose Estimation in the Wildcited by 131

50 most cited of 80 All 80 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 24; 2026: 91.

117 papers.

Sort:

By area:

  1. 2025HaWoR: World-Space Hand Motion Reconstruction from Egocentric Videoscited by 62
  2. 2026SAM 3D Body: Robust Full-Body Human Mesh Recoverycited by 61
  3. 2026CLAP: Contrastive Latent Action Pretraining for Learning Vision-Language-Action Models from Human Videoscited by 27
  4. 2024Signs as Tokens: A Retrieval-Enhanced Multilingual Sign Language Generatorcited by 20
  5. 2025Precise Action-to-Video Generation Through Visual Action Promptscited by 15
  6. 2026HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videoscited by 14
  7. 2025PhySIC: Physically Plausible 3D Human-Scene Interaction and Contact from a Single Imagecited by 13
  8. 2025HORT: Monocular Hand-held Objects Reconstruction with Transformerscited by 11
  9. 2025Follow My Hold: Hand-Object Interaction Reconstruction through Geometric Guidancecited by 8
  10. 2026Joint-Aligned Latent Action: Towards Scalable VLA Pretraining in the Wildcited by 8
  11. 2026ACE-Ego-0: Unifying Egocentric Human and Robotic Data for VLA Pretrainingcited by 8
  12. 2025Perceiving and Acting in First-Person: A Dataset and Benchmark for Egocentric Human-Object-Human Interactionscited by 7
38 more
  1. 2025EgoEdit: Dataset, Real-Time Streaming Model, and Benchmark for Egocentric Video Editingcited by 7
  2. 2026EgoGrasp: World-Space Hand-Object Interaction Estimation from Egocentric Videoscited by 7
  3. 2026FunRec: Reconstructing Functional 3D Scenes from Egocentric Interaction Videoscited by 7
  4. 20263D-Aware Implicit Motion Control for View-Adaptive Human Video Generationcited by 6
  5. 2026ForeHOI: Feed-forward 3D Object Reconstruction from Daily Hand-Object Interaction Videoscited by 6
  6. 2026FlowHOI: Flow-based Semantics-Grounded Generation of Hand-Object Interactions for Dexterous Robot Manipulationcited by 6
  7. 2024MaskHand: Generative Masked Modeling for Robust Hand Mesh Reconstruction in the Wildcited by 5
  8. 2025Diffusion-based 3D Hand Motion Recovery with Intuitive Physicscited by 5
  9. 2025Text-Driven 3D Hand Motion Generation from Sign Language Datacited by 5
  10. 2025LuciBot: Automated Robot Policy Learning from Generated Videoscited by 4
  11. 2025ImHead: A Large-scale Implicit Morphable Model for Localized Head Modelingcited by 4
  12. 2026AGILE: Hand-Object Interaction Reconstruction from Video via Agentic Generationcited by 4
  13. 2026Fast SAM 3D Body: Accelerating SAM 3D Body for Real-Time Full-Body Human Mesh Recoverycited by 4
  14. 2026PAD-Hand: Physics-Aware Diffusion for Hand Motion Recoverycited by 4
  15. 2026The Surprising Effectiveness of Video Diffusion Models for Hand Motion Reconstructioncited by 4
  16. 2025Efficient and Scalable Monocular Human-Object Interaction Motion Reconstructioncited by 3
  17. 2026Controllable Egocentric Video Generation via Occlusion-Aware Sparse 3D Hand Jointscited by 3
  18. 2026PAM: A Pose-Appearance-Motion Engine for Sim-to-Real HOI Video Generationcited by 3
  19. 2026HGGT: Robust and Flexible 3D Hand Mesh Reconstruction from Uncalibrated Imagescited by 3
  20. 2026ArtHOI: Taming Foundation Models for Monocular 4D Reconstruction of Hand-Articulated-Object Interactionscited by 3
  21. 2026EgoForce: Forearm-Guided Camera-Space 3D Hand Pose from a Monocular Egocentric Cameracited by 3
  22. 2025Affordance-Guided Diffusion Prior for 3D Hand Reconstructioncited by 2
  23. 2025DeLTa: Demonstration and Language-Guided Novel Transparent Object Manipulationcited by 2
  24. 2025HanDyVQA: A Video QA Benchmark for Fine-Grained Hand-Object Interaction Dynamicscited by 2
  25. 2025Reconstructing Objects along Hand Interaction Timelines in Egocentric Videocited by 2
  26. 2026EgoHandICL: Egocentric 3D Hand Reconstruction with In-Context Learningcited by 2
  27. 2026Open-Vocabulary Functional 3D Human-Scene Interaction Generationcited by 2
  28. 2026UniHand: A Unified Model for Diverse Controlled 4D Hand Motion Modelingcited by 2
  29. 2026SignSparK: Efficient Multilingual Sign Language Production via Sparse Keyframe Learningcited by 2
  30. 2026GHOST: Fast Category-agnostic Hand-Object Interaction Reconstruction from RGB Videos using Gaussian Splattingcited by 2
  31. 2026M3T: Discrete Multi-Modal Motion Tokens for Sign Language Productioncited by 2
  32. 2026MobileEgo Anywhere: Open Infrastructure for long horizon egocentric data on commodity hardwarecited by 2
  33. 2026StableHand: Quality-Aware Flow Matching for World-Space Dual-Hand Motion Estimation from Egocentric Videocited by 2
  34. 2026Open-CHOIR: Open-World Contact-Aware 4D Hand-Object Interaction Reconstructioncited by 2
  35. 2026HARP-VLA: Human-Robot Aligned Representation Learning for Vision-Language-Action Modelcited by 2
  36. 2026What Matters When Cotraining Robot Manipulation Policies on Everyday Human Videos?cited by 2
  37. 2026GHOST: Hierarchical Sub-Goal Policies for Generalizing Robot Manipulationcited by 2
  38. 2026LUCID: Learning Embodiment-Agnostic Intent Models from Unstructured Human Videos for Scalable Dexterous Robot Skill Acquisitioncited by 2

50 most cited of 117 All 117 (JSON)