mlsanityData to 7 Oct 2026

HandGCAT: Occlusion-Robust 3D Hand Mesh Reconstruction from Monocular Images

arXiv 2403.07912 ↗First posted 27 Feb 2024cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

We propose a robust and accurate method for reconstructing 3D hand mesh from monocular images. This is a very challenging problem, as hands are often severely occluded by objects. Previous works often have disregarded 2D hand pose information, which contains hand prior knowledge that is strongly correlated with occluded regions. Thus, in this work, we propose a novel 3D hand mesh reconstruction network HandGCAT, that can fully exploit hand prior as compensation information to enhance occluded region features. Specifically, we designed the Knowledge-Guided Graph Convolution (KGC) module and the Cross-Attention Transformer (CAT) module. KGC extracts hand prior information from 2D hand pose by graph convolution. CAT fuses hand prior into occluded regions by considering their high correlation. Extensive experiments on popular datasets with challenging hand-object occlusions, such as HO3D v2, HO3D v3, and DexYCB demonstrate that our HandGCAT reaches state-of-the-art performance. The code is available at https://github.com/heartStrive/HandGCAT.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 30 of its 39 references that are in the map. First posted on arXiv 27 Feb 2024.

Builds on

30 of 39 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,085
  3. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  4. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 913
  5. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 417
  6. 2016RMPE: Regional Multi-person Pose Estimationcited by 340
  7. 2021DexYCB: A Benchmark for Capturing Hand Grasping of Objectscited by 320
  8. 2020I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Imagecited by 274
  9. 2021Mesh Graphormercited by 264
  10. 2020Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Posecited by 248
  11. 20193D Hand Shape and Pose Estimation from a Single RGB Imagecited by 174
  12. 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 140
18 more
  1. 2020Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstructioncited by 139
  2. 2021Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimationcited by 122
  3. 2020Weakly Supervised 3D Hand Pose Estimation via Biomechanical Constraintscited by 97
  4. 2022HandOccNet: Occlusion-Robust 3D Hand Mesh Estimation Networkcited by 95
  5. 2021ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesiscited by 76
  6. 2021MobRecon: Mobile-Friendly Hand Mesh Reconstruction from Monocular Imagecited by 74
  7. 20203D Human Pose Estimation using Spatio-Temporal Networks with Explicit Occlusion Trainingcited by 54
  8. 2022Collaborative Learning for Hand and Object Reconstruction with Attention-guided Graph Convolutioncited by 54
  9. 2021I2UV-HandNet: Image-to-UV Prediction Network for Accurate and High-fidelity 3D Hand Mesh Modelingcited by 52
  10. 2018Multi-Scale Structure-Aware Network for Human Pose Estimationcited by 37
  11. 2022TSA-Net: Tube Self-Attention Network for Action Quality Assessmentcited by 36
  12. 2023Context De-confounded Emotion Recognitioncited by 34
  13. 2021HO-3D_v3: Improving the Accuracy of Hand-Object Annotations of the HO-3D Datasetcited by 25
  14. 2018How Robust is 3D Human Pose Estimation to Occlusion?cited by 23
  15. 2022A Survey of Video-based Action Quality Assessmentcited by 14
  16. 2022CA-SpaceNet: Counterfactual Analysis for 6D Pose Estimation in Spacecited by 13
  17. 2023Towards Simultaneous Segmentation of Liver Tumors and Intrahepatic Vessels via Cross-attention Mechanismcited by 8
  18. 2023Out of Thin Air: Exploring Data-Free Adversarial Robustness Distillationcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 0; 2026: 1.

4 papers.

  1. 2024Hamba: Single-view 3D Hand Reconstruction with Graph-guided Bi-Scanning Mambacited by 45
  2. 2024MaskHand: Generative Masked Modeling for Robust Hand Mesh Reconstruction in the Wildcited by 5
  3. 2024HandS3C: 3D Hand Mesh Reconstruction with State Space Spatial Channel Attention from RGB imagescited by 0
  4. 2026ScaleHP: Scale-Mediated Optimization of Coupled Errors for Metric-Space Hand Pose Estimationcited by 0