mlsanityData to 14 Oct 2026

HandDiff: 3D Hand Pose Estimation with Diffusion on Image-Point Cloud

arXiv 2404.03159 ↗First posted 4 Apr 2024cs.CVcited by 12

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

Extracting keypoint locations from input hand frames, known as 3D hand pose estimation, is a critical task in various human-computer interaction applications. Essentially, the 3D hand pose estimation can be regarded as a 3D point subset generative problem conditioned on input frames. Thanks to the recent significant progress on diffusion-based generative models, hand pose estimation can also benefit from the diffusion model to estimate keypoint locations with high quality. However, directly deploying the existing diffusion models to solve hand pose estimation is non-trivial, since they cannot achieve the complex permutation mapping and precise localization. Based on this motivation, this paper proposes HandDiff, a diffusion-based hand pose estimation model that iteratively denoises accurate hand pose conditioned on hand-shaped image-point clouds. In order to recover keypoint permutation and accurate location, we further introduce joint-wise condition and local detail condition. Experimental results demonstrate that the proposed HandDiff significantly outperforms the existing approaches on four challenging hand pose benchmark datasets. Codes and pre-trained models are publicly available at https://github.com/cwc1260/HandDiff.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 39 of its 55 references that are in the map. First posted on arXiv 4 Apr 2024.

Builds on

39 of 55 references.

References · 39

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  5. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  6. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  7. 2021Image Super-Resolution via Iterative Refinementcited by 1,511
  8. 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
  9. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
  10. 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 735
  11. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 553
  12. 2022DiGress: Discrete Denoising diffusion for graph generationcited by 520
27 more
  1. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  2. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 499
  3. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 418
  4. 2022DiffusionDet: Diffusion Model for Object Detectioncited by 354
  5. 2021DexYCB: A Benchmark for Capturing Hand Grasping of Objectscited by 325
  6. 2022Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equationscited by 283
  7. 2020Permutation Invariant Graph Generation via Score-Based Generative Modelingcited by 270
  8. 2018FlowNet3D: Learning Scene Flow in 3D Point Cloudscited by 242
  9. 2017V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Mapcited by 102
  10. 2021A Conditional Point Diffusion-Refinement Paradigm for 3D Point Cloud Completioncited by 100
  11. 2022DiffPose: Toward More Reliable 3D Pose Estimationcited by 100
  12. 2020Weakly Supervised 3D Hand Pose Estimation via Biomechanical Constraintscited by 97
  13. 2022HandOccNet: Occlusion-Robust 3D Hand Mesh Estimation Networkcited by 96
  14. 2023Diffusion-Based 3D Human Pose Estimation with Multi-Hypothesis Aggregationcited by 82
  15. 2022DiffPose: Multi-hypothesis Human Pose Estimation using Diffusion modelscited by 65
  16. 2019A2J: Anchor-to-Joint Regression Network for 3D Articulated Pose Estimation from a Single Depth Imagecited by 56
  17. 2022Collaborative Learning for Hand and Object Reconstruction with Attention-guided Graph Convolutioncited by 54
  18. 2016Robust 3D Hand Pose Estimation in Single Depth Images: from Single-View CNN to Multi-View CNNscited by 29
  19. 2022DiffuPose: Monocular 3D Human Pose Estimation via Denoising Diffusion Probabilistic Modelcited by 29
  20. 2018Point-to-Pose Voting based Hand Pose Estimation using Residual Permutation Equivariant Layercited by 21
  21. 2020JGR-P2O: Joint Graph Reasoning based Pixel-to-Offset Prediction Network for 3D Hand Pose Estimation from a Single Depth Imagecited by 20
  22. 2017DeepPrior++: Improving Fast and Accurate 3D Hand Pose Estimationcited by 19
  23. 2017Dense 3D Regression for Hand Pose Estimationcited by 18
  24. 2021HandFoldingNet: A 3D Hand Pose Estimation Network Using Multiscale-Feature Guided Folding of a 2D Hand Skeletoncited by 16
  25. 2017Pose Guided Structured Region Ensemble Network for Cascaded Hand Pose Estimationcited by 12
  26. 2017Region Ensemble Network: Improving Convolutional Network for Hand Pose Estimationcited by 11
  27. 2022Efficient Virtual View Selection for 3D Hand Pose Estimationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 7; 2026: 5.

12 papers.

  1. 2025Mitty: Diffusion-based Human-to-Robot Video Generationcited by 14
  2. 2025HOSt3R: Keypoint-free Hand-Object 3D Reconstruction from RGB imagescited by 3
  3. 2026AnyHand: A Large-Scale Synthetic Dataset for RGB(-D) Hand Pose Estimationcited by 2
  4. 2026HandFlow: Fully Generative 4D Hand Recovery with Flow Matchingcited by 2
  5. 2025Bimanual 3D Hand Motion and Articulation Forecasting in Everyday Imagescited by 1
  6. 2025Enhancing Monocular 3D Hand Reconstruction with Learned Texture Priorscited by 0
  7. 2025Cascaded Diffusion Framework for Probabilistic Coarse-to-Fine Hand Pose Estimationcited by 0
  8. 2025KineDiff3D: Kinematic-Aware Diffusion for Category-Level Articulated Object Shape Reconstruction and Generationcited by 0
  9. 2025Point2Pose: A Generative Framework for 3D Human Pose Estimation with Multi-View Point Cloud Datasetcited by 0
  10. 2026HandMCM: Multi-modal Point Cloud-based Correspondence State Space Model for 3D Hand Pose Estimationcited by 0
  11. 2026AirGlove: Exploring Egocentric 3D Hand Tracking and Appearance Generalization for Sensing Glovescited by 0
  12. 2026KPGrasp: Scalable Keypoint Flow Matching for Dexterous Grasp Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Denoising Diffusion for 3D Hand Pose Estimation from Images
  2. 2024HOIDiffusion: Generating Realistic 3D Hand-Object Interaction Data
  3. 2024ManiDext: Hand-Object Manipulation Synthesis via Continuous Correspondence Embeddings and Residual-Guided Diffusion
  4. 2024FastGrasp: Efficient Grasp Synthesis with Diffusion
  5. 2025HanDrawer: Leveraging Spatial Information to Render Realistic Hands Using a Conditional Diffusion Model in Single Stage
  6. 2026The Surprising Effectiveness of Video Diffusion Models for Hand Motion Reconstruction
  7. 20253D Hand Mesh-Guided AI-Generated Malformed Hand Refinement with Hand Pose Transformation via Diffusion Model
  8. 2023DiffHand: End-to-End Hand Mesh Reconstruction via Diffusion Models