mlsanityData to 7 Oct 2026

Identity-Aware Hand Mesh Estimation and Personalization from RGB Images

arXiv 2209.10840 ↗First posted 22 Sep 2022cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

Reconstructing 3D hand meshes from monocular RGB images has attracted increasing amount of attention due to its enormous potential applications in the field of AR/VR. Most state-of-the-art methods attempt to tackle this task in an anonymous manner. Specifically, the identity of the subject is ignored even though it is practically available in real applications where the user is unchanged in a continuous recording session. In this paper, we propose an identity-aware hand mesh estimation model, which can incorporate the identity information represented by the intrinsic shape parameters of the subject. We demonstrate the importance of the identity information by comparing the proposed identity-aware model to a baseline which treats subject anonymously. Furthermore, to handle the use case where the test subject is unseen, we propose a novel personalization pipeline to calibrate the intrinsic shape parameters using only a few unlabeled RGB images of the subject. Experiments on two large scale public datasets validate the state-of-the-art performance of our proposed method.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 42 of its 57 references that are in the map. First posted on arXiv 22 Sep 2022.

Builds on

42 of 57 references.

References · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2020End-to-End Object Detection with Transformerscited by 7,384
  8. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  9. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
  10. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 913
  11. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 417
  12. 2019Learning joint reconstruction of hands and manipulated objectscited by 377
30 more
  1. 2021DexYCB: A Benchmark for Capturing Hand Grasping of Objectscited by 320
  2. 2020I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Imagecited by 274
  3. 2019FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Imagescited by 271
  4. 2021Mesh Graphormercited by 264
  5. 2020InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Imagecited by 244
  6. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  7. 2017Hand Keypoint Detection in Single Images using Multiview Bootstrappingcited by 231
  8. 20193D Hand Shape and Pose from Images in the Wildcited by 190
  9. 2022Embodied Hands: Modeling and Capturing Hands and Bodies Togethercited by 179
  10. 20193D Hand Shape and Pose Estimation from a Single RGB Imagecited by 174
  11. 2020Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wildcited by 146
  12. 2019End-to-end Hand Mesh Recovery from a Monocular RGB Imagecited by 124
  13. 2021Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimationcited by 122
  14. 2019Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Renderingcited by 115
  15. 2017V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Mapcited by 102
  16. 2020Weakly Supervised 3D Hand Pose Estimation via Biomechanical Constraintscited by 97
  17. 2022HandOccNet: Occlusion-Robust 3D Hand Mesh Estimation Networkcited by 95
  18. 2021Camera-Space Hand Mesh Recovery via Semantic Aggregation and Adaptive 2D-1D Registrationcited by 59
  19. 2020DeepHandMesh: A Weakly-supervised Deep Encoder-Decoder Framework for High-fidelity Hand Mesh Modelingcited by 49
  20. 2021AFTer-UNet: Axial Fusion Transformer UNet for Medical Image Segmentationcited by 48
  21. 2018A Simple Approach to Intrinsic Correspondence Learning on Unstructured 3D Meshescited by 39
  22. 2020Predicting Camera Viewpoint Improves Cross-dataset Generalization for 3D Human Pose Estimationcited by 36
  23. 2021TransFusion: Cross-view Fusion with Transformer for 3D Human Pose Estimationcited by 32
  24. 2022PPT: token-Pruned Pose Transformer for monocular and multi-view human pose estimationcited by 27
  25. 2020BiHand: Recovering Hand Mesh with Multi-stage Bisected Hourglass Networkscited by 23
  26. 2020Nonparametric Structure Regularization Machine for 2D Hand Pose Estimationcited by 12
  27. 2019Adaptive Graphical Model Network for 2D Handpose Estimationcited by 9
  28. 2020SIA-GCN: A Spatial Information Aware Graph Neural Network with 2D Convolutions for Hand Pose Estimationcited by 9
  29. 2022The Best of Both Worlds: Combining Model-based and Nonparametric Approaches for 3D Human Body Estimationcited by 8
  30. 2020Rotation-invariant Mixed Graphical Model Network for 2D Hand Pose Estimationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 5; 2025: 0; 2026: 1.

11 papers.

  1. 2023POEM: Reconstructing Hand in a Point Embedded Multi-view Stereocited by 9
  2. 2023Decoupled Iterative Refinement Framework for Interacting Hands Reconstruction from a Single RGB Imagecited by 8
  3. 2023DiffHand: End-to-End Hand Mesh Reconstruction via Diffusion Modelscited by 7
  4. 2024Multi-view Hand Reconstruction with a Point-Embedded Transformercited by 6
  5. 2023PHRIT: Parametric Hand Representation with Implicit Templatecited by 5
  6. 2024MLPHand: Real Time Multi-View 3D Hand Mesh Reconstruction via MLP Modelingcited by 3
  7. 2024Fine-Grained Multi-View Hand Reconstruction Using Inverse Renderingcited by 3
  8. 2026HGGT: Robust and Flexible 3D Hand Mesh Reconstruction from Uncalibrated Imagescited by 3
  9. 2024XHand: Real-time Expressive Hand Avatarcited by 2
  10. 2023Self-supervised Optimization of Hand Pose Estimation using Anatomical Features and Iterative Learningcited by 0
  11. 20243D Hand Mesh Recovery from Monocular RGB in Camera Spacecited by 0