mlsanityData to 7 Oct 2026

SiMA-Hand: Boosting 3D Hand-Mesh Reconstruction by Single-to-Multi-View Adaptation

arXiv 2402.01389 ↗First posted 2 Feb 2024cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Estimating 3D hand mesh from RGB images is a longstanding track, in which occlusion is one of the most challenging problems. Existing attempts towards this task often fail when the occlusion dominates the image space. In this paper, we propose SiMA-Hand, aiming to boost the mesh reconstruction performance by Single-to-Multi-view Adaptation. First, we design a multi-view hand reconstructor to fuse information across multiple views by holistically adopting feature fusion at image, joint, and vertex levels. Then, we introduce a single-view hand reconstructor equipped with SiMA. Though taking only one view as input at inference, the shape and orientation features in the single-view reconstructor can be enriched by learning non-occluded knowledge from the extra views at training, enhancing the reconstruction precision on the occluded regions. We conduct experiments on the Dex-YCB and HanCo benchmarks with challenging object- and self-caused occlusion cases, manifesting that SiMA-Hand consistently achieves superior performance over the state of the arts. Code will be released on https://github.com/JoyboyWang/SiMA-Hand Pytorch.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 41 of its 59 references that are in the map. First posted on arXiv 2 Feb 2024.

Builds on

41 of 59 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  4. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  5. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
  6. 2019Similarity-Preserving Knowledge Distillationcited by 507
  7. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 417
  8. 2019Learning joint reconstruction of hands and manipulated objectscited by 377
  9. 2019A Comprehensive Overhaul of Feature Distillationcited by 341
  10. 2021DexYCB: A Benchmark for Capturing Hand Grasping of Objectscited by 320
  11. 2019Structured Knowledge Distillation for Dense Predictioncited by 277
  12. 2019FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Imagescited by 271
29 more
  1. 2020InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Imagecited by 244
  2. 20193D Hand Shape and Pose from Images in the Wildcited by 190
  3. 20193D Hand Shape and Pose Estimation from a Single RGB Imagecited by 174
  4. 2021Hand-Object Contact Consistency Reasoning for Human Grasps Generationcited by 160
  5. 2020Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wildcited by 146
  6. 2020Reconstructing Hand-Object Interactions in the Wildcited by 143
  7. 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 140
  8. 2020Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstructioncited by 139
  9. 2019End-to-end Hand Mesh Recovery from a Monocular RGB Imagecited by 124
  10. 2022Cross-Image Relational Knowledge Distillation for Semantic Segmentationcited by 117
  11. 2018Hand Pose Estimation via Latent 2.5D Heatmap Regressioncited by 111
  12. 2020Monocular Real-time Hand Shape and Motion Capture using Multi-modal Datacited by 106
  13. 2021Distilling Object Detectors via Decoupled Featurescited by 105
  14. 2022HandOccNet: Occlusion-Robust 3D Hand Mesh Estimation Networkcited by 95
  15. 2021MobRecon: Mobile-Friendly Hand Mesh Reconstruction from Monocular Imagecited by 74
  16. 2021Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Cameracited by 70
  17. 2017Multi-Context Attention for Human Pose Estimationcited by 66
  18. 2022LISA: Learning Implicit Shape and Appearance of Handscited by 63
  19. 2021Camera-Space Hand Mesh Recovery via Semantic Aggregation and Adaptive 2D-1D Registrationcited by 59
  20. 2022Collaborative Learning for Hand and Object Reconstruction with Attention-guided Graph Convolutioncited by 54
  21. 2018A Simple Approach to Intrinsic Correspondence Learning on Unstructured 3D Meshescited by 39
  22. 2022Hierarchical Temporal Transformer for 3D Hand Pose Estimation and Action Recognition from Egocentric RGB Videoscited by 35
  23. 2020SeqHAND:RGB-Sequence-Based 3D Hand Pose and Shape Estimationcited by 34
  24. 2021Contrastive Representation Learning for Hand Shape Estimationcited by 34
  25. 2021Multi-Scale Aligned Distillation for Low-Resolution Detectioncited by 24
  26. 2020HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Mapcited by 23
  27. 2020BiHand: Recovering Hand Mesh with Multi-stage Bisected Hourglass Networkscited by 23
  28. 2018RGB-based 3D Hand Pose Estimation via Privileged Learning with Depth Imagescited by 10
  29. 2019Dual Grid Net: hand mesh vertex regression from single depth mapscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026EPS3D: End-to-End Feed-Forward 3D Panoptic Segmentationcited by 2
  2. 2026CNS-Edit++: Category-Agnostic 3D Editing with Coupled Neural Shape Representationcited by 0