mlsanityData to 14 Oct 2026

Authentic Hand Avatar from a Phone Scan via Universal Hand Model

arXiv 2405.07933 ↗First posted 13 May 2024cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

The authentic 3D hand avatar with every identifiable information, such as hand shapes and textures, is necessary for immersive experiences in AR/VR. In this paper, we present a universal hand model (UHM), which 1) can universally represent high-fidelity 3D hand meshes of arbitrary identities (IDs) and 2) can be adapted to each person with a short phone scan for the authentic hand avatar. For effective universal hand modeling, we perform tracking and modeling at the same time, while previous 3D hand models perform them separately. The conventional separate pipeline suffers from the accumulated errors from the tracking stage, which cannot be recovered in the modeling stage. On the other hand, ours does not suffer from the accumulated errors while having a much more concise overall pipeline. We additionally introduce a novel image matching loss function to address a skin sliding during the tracking and modeling, while existing works have not focused on it much. Finally, using learned priors from our UHM, we effectively adapt our UHM to each person's short phone scan for the authentic hand avatar.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 33 of its 45 references that are in the map. First posted on arXiv 13 May 2024.

Builds on

33 of 45 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  5. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,908
  6. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  7. 2018Group Normalizationcited by 1,524
  8. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  9. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
  10. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 950
  11. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 575
  12. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 418
21 more
  1. 2020I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Imagecited by 274
  2. 2021Mesh Graphormercited by 265
  3. 2020Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Posecited by 248
  4. 2020InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Imagecited by 245
  5. 2020STAR: Sparse Trained Articulated Human Body Regressorcited by 194
  6. 20193D Hand Shape and Pose Estimation from a Single RGB Imagecited by 174
  7. 2017Learning to Predict Indoor Illumination from a Single Imagecited by 161
  8. 2020MediaPipe Hands: On-device Real-time Hand Trackingcited by 161
  9. 2021Robust High-Resolution Video Matting with Temporal Guidancecited by 137
  10. 2020Accurate 3D Hand Pose Estimation for Whole-Body 3D Human Mesh Estimationcited by 81
  11. 2021Driving-Signal Aware Full-Body Avatarscited by 81
  12. 2019Deep Sky Modeling for Single Image Outdoor Lighting Estimationcited by 66
  13. 2022LISA: Learning Implicit Shape and Appearance of Handscited by 63
  14. 2020NeuralAnnot: Neural Annotator for 3D Human Mesh Training Setscited by 62
  15. 2020DeepHandMesh: A Weakly-supervised Deep Encoder-Decoder Framework for High-fidelity Hand Mesh Modelingcited by 49
  16. 2022Hand Avatar: Free-Pose Hand Animation and Rendering from Monocular Videocited by 37
  17. 2023Bringing Inputs to Shared Domains for 3D Interacting Hands Recovery in the Wildcited by 36
  18. 2022HARP: Personalized Hand Reconstruction from a Monocular RGB Videocited by 28
  19. 2023LiveHand: Real-time and Photorealistic Neural Hand Renderingcited by 26
  20. 2023RelightableHands: Efficient Neural Relighting of Articulated Hand Modelscited by 21
  21. 2024URHand: Universal Relightable Handscited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 7.

13 papers.

Sort:

  1. 2024Expressive Whole-Body 3D Gaussian Avatarcited by 89
  2. 2025Vid2Avatar-Pro: Authentic Avatar from Videos in the Wild via Universal Priorcited by 31
  3. 2025Parametric Gaussian Human Model: Generalizable Prior for Efficient and Realistic Human Avatar Modelingcited by 3
  4. 2025SRHand: Super-Resolving Hand Images and 3D Shapes via View/Pose-aware Neural Image Representations and Explicit 3D Meshescited by 3
  5. 2025PALM: A Dataset and Baseline for Learning Multi-subject Hand Priorcited by 3
  6. 2026Enhancing Hands in 3D Whole-Body Pose Estimation with Conditional Hands Modulatorcited by 2
  7. 2025HandSCS: Structural Coordinate Space for Animatable Hand Gaussian Splattingcited by 1
  8. 2026WildGHand: Learning Anti-Perturbation Gaussian Hand Avatars from Monocular In-the-Wild Videoscited by 0
  9. 2026PHAF-Personalized Hand Avatars in a Flashcited by 0
  10. 2026Self-Learning Expression Deformations for Data-Efficient Gaussian Avatarscited by 0
  11. 2026VEPHand: View-Efficient Photometric Hand Performance Capture at Scalecited by 0
  12. 2026Hand-4DGS: Feed-Forward 3D Gaussian Splatting for 4D Hand Reconstruction from Egocentric Videoscited by 0
1 more
  1. 2026OASIS: Occlusion-aware Single-image Hand Avatar Reconstruction via 3D Gaussian Splattingcited by 0