mlsanityData to 14 Oct 2026

HARP: Personalized Hand Reconstruction from a Monocular RGB Video

arXiv 2212.09530 ↗First posted 19 Dec 2022cs.CVcited by 28

Not on the map. Why?

Abstract (arXiv)

We present HARP (HAnd Reconstruction and Personalization), a personalized hand avatar creation approach that takes a short monocular RGB video of a human hand as input and reconstructs a faithful hand avatar exhibiting a high-fidelity appearance and geometry. In contrast to the major trend of neural implicit representations, HARP models a hand with a mesh-based parametric hand model, a vertex displacement map, a normal map, and an albedo without any neural components. As validated by our experiments, the explicit nature of our representation enables a truly scalable, robust, and efficient approach to hand avatar creation. HARP is optimized via gradient descent from a short sequence captured by a hand-held mobile phone and can be directly used in AR/VR applications with real-time rendering capability. To enable this, we carefully design and implement a shadow-aware differentiable rendering scheme that is robust to high degree articulations and self-shadowing regularly present in hand motion sequences, as well as challenging lighting conditions. It also generalizes to unseen poses and novel viewpoints, producing photo-realistic renderings of hand animations performing highly-articulated motions. Furthermore, the learned HARP representation can be used for improving 3D hand pose estimation quality in challenging viewpoints. The key advantages of HARP are validated by the in-depth analyses on appearance reconstruction, novel-view and novel pose synthesis, and 3D hand pose refinement. It is an AR/VR-ready personalized hand representation that shows superior fidelity and scalability.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 68 of its 92 references that are in the map. First posted on arXiv 19 Dec 2022.

Builds on

68 of 92 references.

References · 68

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  5. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,908
  6. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  7. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,198
  8. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 784
  9. 2020First Order Motion Model for Image Animationcited by 704
  10. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  11. 2022HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Videocited by 483
  12. 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 477
38 more
  1. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 467
  2. 2017Neural 3D Mesh Renderercited by 437
  3. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  4. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 418
  5. 2019Learning joint reconstruction of hands and manipulated objectscited by 378
  6. 2021DexYCB: A Benchmark for Capturing Hand Grasping of Objectscited by 325
  7. 2020I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Imagecited by 274
  8. 2019Convolutional Mesh Regression for Single-Image Human Shape Reconstructioncited by 270
  9. 2018Deep Video Portraitscited by 267
  10. 2020Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Posecited by 248
  11. 2020InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Imagecited by 245
  12. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  13. 2017First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotationscited by 230
  14. 2021HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fieldscited by 214
  15. 2021Neural Head Avatars from Monocular RGB Videoscited by 213
  16. 2021Neural Articulated Radiance Fieldcited by 196
  17. 2022Embodied Hands: Modeling and Capturing Hands and Bodies Togethercited by 181
  18. 2020Grasping Field: Learning Implicit Representations for Human Graspscited by 179
  19. 20193D Hand Shape and Pose Estimation from a Single RGB Imagecited by 174
  20. 2022TAVA: Template-free Animatable Volumetric Actorscited by 172
  21. 2022Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoisingcited by 163
  22. 2021H-NeRF: Neural Radiance Fields for Rendering and Temporal Reconstruction of Humans in Motioncited by 157
  23. 2017GANerated Hands for Real-time 3D Hand Tracking from Monocular RGBcited by 147
  24. 2020Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wildcited by 146
  25. 2020Reconstructing Hand-Object Interactions in the Wildcited by 144
  26. 2019H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Interactionscited by 141
  27. 2021Robust High-Resolution Video Matting with Temporal Guidancecited by 137
  28. 2021NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illuminationcited by 135
  29. 2020HOPE-Net: A Graph-based Model for Hand-Object Pose Estimationcited by 112
  30. 2018Hand Pose Estimation via Latent 2.5D Heatmap Regressioncited by 111
  31. 2021LEAP: Learning Articulated Occupancy of Peoplecited by 107
  32. 2017V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Mapcited by 102
  33. 2020Weakly Supervised 3D Hand Pose Estimation via Biomechanical Constraintscited by 97
  34. 2021MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Imagescited by 78
  35. 2022KeypointNeRF: Generalizing Image-based Volumetric Avatars using Relative Spatial Encoding of Keypointscited by 77
  36. 2021Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Cameracited by 70
  37. 2020SMPLpix: Neural Avatars from 3D Human Modelscited by 69
  38. 2021Model-based 3D Hand Reconstruction via Self-Supervised Learningcited by 65

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 9; 2025: 7; 2026: 7.

28 papers.

Sort:

  1. 2023An Outlook into the Future of Egocentric Visioncited by 84
  2. 2023MANUS: Markerless Grasp Capture using Articulated 3D Gaussianscited by 26
  3. 2024Reconstructing Hand-Held Objects in 3D from Images and Videoscited by 24
  4. 2024EgoPressure: A Dataset for Hand Pressure and Pose Estimation in Egocentric Visioncited by 19
  5. 2024OHTA: One-shot Hand Avatar via Data-driven Implicit Priorscited by 18
  6. 2024URHand: Universal Relightable Handscited by 17
  7. 2024Authentic Hand Avatar from a Phone Scan via Universal Hand Modelcited by 13
  8. 20233D Points Splatting for Real-Time Dynamic Hand Reconstructioncited by 10
  9. 2024BiTT: Bi-directional Texture Reconstruction of Interacting Two Hands from a Single Imagecited by 8
  10. 2025DyTact: Capturing Dynamic Contacts in Hand-Object Manipulationcited by 8
  11. 2025Advances in Radiance Field for Dynamic Scene: From Neural Field to Gaussian Fieldcited by 5
  12. 2024Fine-Grained Multi-View Hand Reconstruction Using Inverse Renderingcited by 3
16 more
  1. 2024Learning Interaction-aware 3D Gaussian Splatting for One-shot Hand Avatarscited by 3
  2. 2025SRHand: Super-Resolving Hand Images and 3D Shapes via View/Pose-aware Neural Image Representations and Explicit 3D Meshescited by 3
  3. 2025PALM: A Dataset and Baseline for Learning Multi-subject Hand Priorcited by 3
  4. 2026Glove2Hand: Synthesizing Natural Hand-Object Interaction from Multi-Modal Sensing Glovescited by 3
  5. 2023Pyramid Deep Fusion Network for Two-Hand Reconstruction from RGB-D Imagescited by 2
  6. 2023HiFiHR: Enhancing 3D Hand Reconstruction from a Single Image via High-Fidelity Texturecited by 2
  7. 2024XHand: Real-time Expressive Hand Avatarcited by 2
  8. 2025HandOcc: NeRF-based Hand Rendering with Occupancy Networkscited by 2
  9. 2025HandSCS: Structural Coordinate Space for Animatable Hand Gaussian Splattingcited by 1
  10. 2025Interaction-Aware 4D Gaussian Splatting for Dynamic Hand-Object Interaction Reconstructioncited by 1
  11. 2026HandDreamer: Zero-Shot Text to 3D Hand Model Generation using Corrective Hand Shape Guidancecited by 0
  12. 2026MASS: Mesh-inellipse Aligned Deformable Surfel Splatting for Hand Reconstruction and Rendering from Egocentric Monocular Videocited by 0
  13. 2026PHAF-Personalized Hand Avatars in a Flashcited by 0
  14. 2026VEPHand: View-Efficient Photometric Hand Performance Capture at Scalecited by 0
  15. 2026Hand-4DGS: Feed-Forward 3D Gaussian Splatting for 4D Hand Reconstruction from Egocentric Videoscited by 0
  16. 2026OASIS: Occlusion-aware Single-image Hand Avatar Reconstruction via 3D Gaussian Splattingcited by 0