mlsanityData to 7 Oct 2026

NIMBLE: A Non-rigid Hand Model with Bones and Muscles

arXiv 2202.04533 ↗First posted 9 Feb 2022cs.CV cs.GRcited by 6

Not on the map. Why?

Abstract (arXiv)

Emerging Metaverse applications demand reliable, accurate, and photorealistic reproductions of human hands to perform sophisticated operations as if in the physical world. While real human hand represents one of the most intricate coordination between bones, muscle, tendon, and skin, state-of-the-art techniques unanimously focus on modeling only the skeleton of the hand. In this paper, we present NIMBLE, a novel parametric hand model that includes the missing key components, bringing 3D hand model to a new level of realism. We first annotate muscles, bones and skins on the recent Magnetic Resonance Imaging hand (MRI-Hand) dataset and then register a volumetric template hand onto individual poses and subjects within the dataset. NIMBLE consists of 20 bones as triangular meshes, 7 muscle groups as tetrahedral meshes, and a skin mesh. Via iterative shape registration and parameter learning, it further produces shape blend shapes, pose blend shapes, and a joint regressor. We demonstrate applying NIMBLE to modeling, rendering, and visual inference tasks. By enforcing the inner bones and muscles to match anatomic and kinematic rules, NIMBLE can animate 3D hands to new poses at unprecedented realism. To model the appearance of skin, we further construct a photometric HandStage to acquire high-quality textures and normal maps to model wrinkles and palm print. Finally, NIMBLE also benefits learning-based hand pose and shape estimation by either synthesizing rich data or acting directly as a differentiable layer in the inference network.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 58 of its 128 references that are in the map. First posted on arXiv 9 Feb 2022.

Builds on

58 of 128 references.

References · 58

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2017Mask R-CNNcited by 7,259
  4. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,417
  5. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  6. 2019Objects as Pointscited by 1,172
  7. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
  8. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  9. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  10. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  11. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  12. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 493
38 more
  1. 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 475
  2. 2019Learning joint reconstruction of hands and manipulated objectscited by 377
  3. 2017Cascaded Pyramid Network for Multi-Person Pose Estimationcited by 341
  4. 2016RMPE: Regional Multi-person Pose Estimationcited by 340
  5. 2020I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Imagecited by 274
  6. 2019FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Imagescited by 271
  7. 2017Integral Human Pose Regressioncited by 255
  8. 2020InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Imagecited by 244
  9. 2016Associative Embedding: End-to-End Learning for Joint Detection and Groupingcited by 211
  10. 2017Single-Shot Multi-Person 3D Pose Estimation From Monocular RGBcited by 195
  11. 2022Embodied Hands: Modeling and Capturing Hands and Bodies Togethercited by 179
  12. 2019Learning Depth-Guided Convolutions for Monocular 3D Object Detectioncited by 169
  13. 2018Bounding Box Regression with Uncertainty for Accurate Object Detectioncited by 143
  14. 2018PersonLab: Person Pose Estimation and Instance Segmentation with a Bottom-Up, Part-Based, Geometric Embedding Modelcited by 133
  15. 2019Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Imagecited by 131
  16. 2015Capturing Hands in Action using Discriminative Salient Points and Physics Simulationcited by 115
  17. 2019Pushing the Envelope for RGB-based Dense 3D Hand Pose Estimation via Neural Renderingcited by 115
  18. 2021Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regressioncited by 112
  19. 2019Self-Supervised Learning of 3D Human Pose using Multi-view Geometrycited by 98
  20. 2019Single-Stage Multi-Person Pose Machinescited by 95
  21. 2018LCR-Net++: Multi-person 2D and 3D Pose Detection in Natural Imagescited by 88
  22. 2019RepNet: Weakly Supervised Training of an Adversarial Reprojection Network for 3D Human Pose Estimationcited by 85
  23. 2021Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Cameracited by 70
  24. 2019XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Cameracited by 68
  25. 2020Weakly-Supervised 3D Human Pose Learning via Multi-view Images in the Wildcited by 66
  26. 2017MonoPerfCap: Human Performance Capture from Monocular Videocited by 63
  27. 2021Learning Skeletal Graph Neural Networks for Hard 3D Pose Estimationcited by 62
  28. 2019Rethinking on Multi-Stage Networks for Human Pose Estimationcited by 58
  29. 2020Differentiable Hierarchical Graph Grouping for Multi-Person Pose Estimationcited by 52
  30. 2020SMAP: Single-Shot Multi-Person Absolute 3D Pose Estimationcited by 52
  31. 2020DeepHandMesh: A Weakly-supervised Deep Encoder-Decoder Framework for High-fidelity Hand Mesh Modelingcited by 49
  32. 2020HMOR: Hierarchical Multi-Person Ordinal Relations for Monocular Multi-Person 3D Pose Estimationcited by 44
  33. 2020Compressed Volumetric Heatmaps for Multi-Person 3D Pose Estimationcited by 36
  34. 2019Single-Network Whole-Body Pose Estimationcited by 24
  35. 2020Graph and Temporal Convolutional Networks for 3D Multi-person Pose Estimation in Monocular Videoscited by 23
  36. 2021PandaNet : Anchor-Based Single-Shot Multi-Person 3D Pose Estimationcited by 22
  37. 2020HDNet: Human Depth Estimation for Multi-Person Camera-Space Localizationcited by 20
  38. 2019Multi-Person 3D Human Pose Estimation from Monocular Imagescited by 19

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 2; 2025: 1.

6 papers.

  1. 2024HOI-M3:Capture Multiple Humans and Objects Interaction within Contextual Environmentcited by 52
  2. 2022Weakly Supervised 3D Multi-person Pose Estimation for Large-scale Scenes based on Monocular Camera and Single LiDARcited by 23
  3. 2024A Unified Diffusion Framework for Scene-aware Human Motion Estimation from Sparse Signalscited by 15
  4. 2023IKOL: Inverse kinematics optimization layer for 3D human pose and shape estimation via Gauss-Newton differentiationcited by 9
  5. 2025ViBES: A Conversational Agent with Behaviorally-Intelligent 3D Virtual Bodycited by 6
  6. 2023Bidirectional GaitNet: A Bidirectional Prediction Model of Human Gait and Anatomical Conditionscited by 3