mlsanityData to 7 Oct 2026

Hand3R: Online 4D Hand-Scene Reconstruction in the Wild

arXiv 2602.03200 ↗First posted 3 Feb 2026cs.CV cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

For Embodied AI, jointly reconstructing dynamic hands and the dense scene context is crucial for understanding physical interaction. However, most existing methods recover isolated hands in local coordinates, overlooking the surrounding 3D environment. To address this, we present Hand3R, the first online framework for joint 4D hand-scene reconstruction from monocular video. Hand3R synergizes a pre-trained hand expert with a 4D scene foundation model via a scene-aware visual prompting mechanism. By injecting high-fidelity hand priors into a persistent scene memory, our approach enables simultaneous reconstruction of accurate hand meshes and dense metric-scale scene geometry in a single forward pass. Experiments demonstrate that Hand3R bypasses the reliance on offline optimization and delivers competitive performance in both local hand reconstruction and global positioning.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 28 of its 29 references that are in the map. First posted on arXiv 3 Feb 2026.

Builds on

28 of 29 references.

References · 28

  1. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 673
  2. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
  3. 2025Continuous 3D Perception Model with Persistent Statecited by 527
  4. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 417
  5. 2019Learning joint reconstruction of hands and manipulated objectscited by 377
  6. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 321
  7. 2021DexYCB: A Benchmark for Capturing Hand Grasping of Objectscited by 320
  8. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 319
  9. 2023Reconstructing Hands in 3D with Transformerscited by 298
  10. 2020I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Imagecited by 274
  11. 2021Mesh Graphormercited by 264
  12. 20193D Hand Shape and Pose from Images in the Wildcited by 190
16 more
  1. 2020Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Videocited by 162
  2. 2021GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Camerascited by 151
  3. 2020Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wildcited by 146
  4. 2021Semi-Supervised 3D Hand-Object Poses Estimation with Interactions in Timecited by 140
  5. 2019End-to-end Hand Mesh Recovery from a Monocular RGB Imagecited by 124
  6. 2024WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wildcited by 117
  7. 2020Weakly Supervised 3D Hand Pose Estimation via Biomechanical Constraintscited by 97
  8. 2022HandOccNet: Occlusion-Robust 3D Hand Mesh Estimation Networkcited by 95
  9. 2025HaWoR: World-Space Hand Motion Reconstruction from Egocentric Videoscited by 62
  10. 2025Human3R: Everyone Everywhere All at Oncecited by 35
  11. 2022The One Where They Reconstructed 3D Humans and Environments in TV Showscited by 30
  12. 2025Joint Optimization for 4D Human-Scene Reconstruction in the Wildcited by 29
  13. 2022Consistent 3D Hand Reconstruction in Video via self-supervised Learningcited by 28
  14. 2023Deformer: Dynamic Fusion Transformer for Robust Hand Pose Estimationcited by 21
  15. 2024Synergistic Global-space Camera and Human Reconstruction from Videoscited by 17
  16. 2025HAMSt3R: Human-Aware Multi-view Stereo 3D Reconstructioncited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Hand-Object Interaction in the Age of Large Foundation Models:Reconstruction, Generation, and Embodied Transfercited by 0