mlsanityData to 14 Oct 2026

Reconstructing Hand-Held Objects from Monocular Video

arXiv 2211.16835 ↗First posted 30 Nov 2022cs.CVcited by 35

Not on the map. Why?

Abstract (arXiv)

This paper presents an approach that reconstructs a hand-held object from a monocular video. In contrast to many recent methods that directly predict object geometry by a trained network, the proposed approach does not require any learned prior about the object and is able to recover more accurate and detailed object geometry. The key idea is that the hand motion naturally provides multiple views of the object and the motion can be reliably estimated by a hand pose tracker. Then, the object geometry can be recovered by solving a multi-view reconstruction problem. We devise an implicit neural representation-based method to solve the reconstruction problem and address the issues of imprecise hand pose estimation, relative hand-object motion, and insufficient geometry optimization for small objects. We also provide a newly collected dataset with 3D ground truth to validate the proposed approach.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Builds on 50 of its 74 references that are in the map. First posted on arXiv 30 Nov 2022.

Builds on

50 of 74 references.

References · 50

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  3. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  4. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  5. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  6. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  7. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  8. 2022TensoRF: Tensorial Radiance Fieldscited by 1,280
  9. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,198
  10. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  11. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 979
  12. 2017End-to-end Recovery of Human Shape and Posecited by 885
38 more
  1. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 788
  2. 2020Implicit Geometric Regularization for Learning Shapescited by 710
  3. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 651
  4. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 642
  5. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 627
  6. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 588
  7. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  8. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 495
  9. 2020Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Videocited by 471
  10. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 418
  11. 2021In-Place Scene Labelling and Understanding with Implicit Scene Representationcited by 415
  12. 2019Learning joint reconstruction of hands and manipulated objectscited by 378
  13. 2016OctNet: Learning Deep 3D Representations at High Resolutionscited by 342
  14. 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 333
  15. 2019FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Imagescited by 271
  16. 2020Neural Unsigned Distance Fields for Implicit Function Learningcited by 265
  17. 2020InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Imagecited by 245
  18. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  19. 2017Hand Keypoint Detection in Single Images using Multiview Bootstrappingcited by 231
  20. 2016Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesiscited by 219
  21. 20193D Hand Shape and Pose from Images in the Wildcited by 190
  22. 2017Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputscited by 189
  23. 2020ContactPose: A Dataset of Grasps with Object Contact and Hand Posecited by 183
  24. 2022Embodied Hands: Modeling and Capturing Hands and Bodies Togethercited by 181
  25. 2020Grasping Field: Learning Implicit Representations for Human Graspscited by 179
  26. 20193D Hand Shape and Pose Estimation from a Single RGB Imagecited by 174
  27. 2020Real-Time High-Resolution Background Mattingcited by 173
  28. 2018Monocular Total Capture: Posing Face, Body, and Hands in the Wildcited by 162
  29. 2017GANerated Hands for Real-time 3D Hand Tracking from Monocular RGBcited by 147
  30. 2020Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wildcited by 146
  31. 2020Learning Canonical Shape Space for Category-Level 6D Object Pose and Size Estimationcited by 124
  32. 2022What's in your hands? 3D Reconstruction of Generic Objects in Handscited by 90
  33. 2018Unsupervised Learning of Shape and Pose with Differentiable Point Cloudscited by 89
  34. 2021Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Cameracited by 70
  35. 20183D-LMNet: Latent Embedding Matching for Accurate and Diverse 3D Point Cloud Reconstruction from a Single Imagecited by 40
  36. 20173D Object Reconstruction from Hand-Object Interactionscited by 31
  37. 2020Two-hand Global 3D Pose Estimation Using Monocular RGBcited by 27
  38. 2017Shape Inpainting using 3D Generative Adversarial Network and Recurrent Convolutional Networkscited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 9; 2024: 9; 2025: 8; 2026: 9.

35 papers.

Sort:

By area:

  1. 2023HOLD: Category-agnostic 3D Reconstruction of Interacting Hands and Objects from Videocited by 78
  2. 2023Diffusion-Guided Reconstruction of Everyday Hand-Object Interaction Clipscited by 74
  3. 2024G-HOP: Generative Hand-Object Prior for Interaction Reconstruction and Grasp Synthesiscited by 43
  4. 2024EasyHOI: Unleashing the Power of Large Models for Reconstructing Hand-Object Interactions in the Wildcited by 36
  5. 2024Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objectscited by 35
  6. 2024Recent Trends in 3D Reconstruction of General Non-Rigid Scenescited by 28
  7. 2024Reconstructing Hand-Held Objects in 3D from Images and Videoscited by 24
  8. 2024InterTrack: Tracking Human Object Interaction without Object Templatescited by 23
  9. 2025MagicHOI: Leveraging 3D Priors for Accurate Hand-object Reconstruction from Short Monocular Video Clipscited by 17
  10. 2023SHOWMe: Benchmarking Object-agnostic Hand-Object 3D Reconstructioncited by 16
  11. 20233D Reconstruction of Objects in Hands without Real World 3D Supervisioncited by 12
  12. 2023MOHO: Learning Single-view Hand-held Object Reconstruction with Multi-view Occlusion-Aware Supervisioncited by 12
23 more
  1. 2025HORT: Monocular Hand-held Objects Reconstruction with Transformerscited by 11
  2. 2023Get a Grip: Reconstructing Hand-Object Stable Grasps in Egocentric Videoscited by 10
  3. 2025How Do I Do That? Synthesizing 3D Hand Motion and Contacts for Everyday Interactionscited by 10
  4. 2026WHOLE: World-Grounded Hand-Object Lifted from Egocentric Videoscited by 8
  5. 2023Color-NeuS: Reconstructing Neural Implicit Surfaces with Colorcited by 7
  6. 2023Ins-HOI: Instance Aware Human-Object Interactions Recoverycited by 7
  7. 2026ForeHOI: Feed-forward 3D Object Reconstruction from Daily Hand-Object Interaction Videoscited by 7
  8. 2025UniHOPE: A Unified Approach for Hand-Only and Hand-Object Pose Estimationcited by 6
  9. 2026Choreographing a World of Dynamic Objectscited by 5
  10. 2023In-Hand 3D Object Reconstruction from a Monocular RGB Videocited by 4
  11. 2026ArtHOI: Taming Foundation Models for Monocular 4D Reconstruction of Hand-Articulated-Object Interactionscited by 4
  12. 2024Free-Moving Object Reconstruction and Pose Estimation with Virtual Cameracited by 3
  13. 2024Benchmarking 2D Egocentric Hand Pose Datasetscited by 3
  14. 2025Towards Dynamic 3D Reconstruction of Hand-Instrument Interaction in Ophthalmic Surgerycited by 3
  15. 2025HOSt3R: Keypoint-free Hand-Object 3D Reconstruction from RGB imagescited by 3
  16. 2025Reconstructing Objects along Hand Interaction Timelines in Egocentric Videocited by 3
  17. 2026Grasp in Gaussians: Fast Monocular Reconstruction of Dynamic Hand-Object Interactionscited by 3
  18. 2025Betsu-Betsu: Multi-View Separable 3D Reconstruction of Two Interacting Objectscited by 2
  19. 2026Towards in-the-wild Egocentric 3D Hand-Object Pose Estimationcited by 2
  20. 2026Hand-Object Interaction in the Age of Large Foundation Models:Reconstruction, Generation, and Embodied Transfercited by 1
  21. 2024PickScan: Object discovery and reconstruction from handheld interactionscited by 0
  22. 2026VEPHand: View-Efficient Photometric Hand Performance Capture at Scalecited by 0
  23. 2026DynamicHOI: Coupled Dynamics for Physics-aware HOI Reconstructioncited by 0