mlsanityData to 7 Oct 2026

Bimanual 3D Hand Motion and Articulation Forecasting in Everyday Images

arXiv 2510.06145 ↗First posted 7 Oct 2025cs.CV cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

We tackle the problem of forecasting bimanual 3D hand motion and articulation from a single image in everyday settings. To address the lack of 3D hand annotations in diverse settings, we design an annotation pipeline consisting of a diffusion model to lift 2D hand keypoint sequences to 4D hand motion. For the forecasting model, we adopt a diffusion loss to account for the multimodality in hand motion distribution. Extensive experiments on 6 datasets show the benefits of training with our imputed labels (14% improvement) and the effectiveness of our lifting (45% better) & forecasting (16.4% gain) models, over the best baselines, especially in zero-shot generalization to everyday images.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 65 of its 79 references that are in the map. First posted on arXiv 7 Oct 2025.

Builds on

65 of 79 references.

References · 65

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,186
  3. 2022Human Motion Diffusion Modelcited by 1,062
  4. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
  5. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 438
  6. 2020End-to-End Human Pose and Mesh Reconstruction with Transformerscited by 417
  7. 2017A simple yet effective baseline for 3d human pose estimationcited by 401
  8. 20183D human pose estimation in video with temporal convolutions and semi-supervised trainingcited by 401
  9. 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 388
  10. 2021DexYCB: A Benchmark for Capturing Hand Grasping of Objectscited by 320
  11. 2022ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulationcited by 306
  12. 2023Reconstructing Hands in 3D with Transformerscited by 298
38 more
  1. 20213D Human Pose Estimation with Spatial and Temporal Transformerscited by 271
  2. 2021H2O: Two Hands Manipulating Objects for First Person Interaction Recognitioncited by 251
  3. 2020InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Imagecited by 244
  4. 2017Learning to Estimate 3D Hand Pose from Single RGB Imagescited by 234
  5. 2023WHAM: Reconstructing World-grounded Humans with Accurate 3D Motioncited by 200
  6. 2017Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistencycited by 177
  7. 2023HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real Worldcited by 159
  8. 2022MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Videocited by 158
  9. 2022PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Imagescited by 152
  10. 2021MHFormer: Multi-Hypothesis Transformer for 3D Human Pose Estimationcited by 151
  11. 2021GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Camerascited by 151
  12. 2020Reconstructing Hand-Object Interactions in the Wildcited by 143
  13. 2023Decoupling Human and Camera Motion from Videos in the Wildcited by 140
  14. 2024Cameras as Rays: Pose Estimation via Ray Diffusioncited by 129
  15. 2021Keypoint Transformer: Solving Joint Identification in Challenging Hands and Object Interactions for Accurate 3D Pose Estimationcited by 122
  16. 2024WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wildcited by 117
  17. 2018Hand Pose Estimation via Latent 2.5D Heatmap Regressioncited by 111
  18. 2022Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videoscited by 110
  19. 2023Ambient Diffusion: Learning Clean Distributions from Corrupted Datacited by 108
  20. 2016Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervisioncited by 106
  21. 2022Ego-Body Pose Estimation via Ego-Head Pose Estimationcited by 93
  22. 2020Motion Guided 3D Pose Estimation from Videoscited by 85
  23. 2024Text2HOI: Text-guided 3D Motion Generation for Hand-Object Interactioncited by 84
  24. 2020Accurate 3D Hand Pose Estimation for Whole-Body 3D Human Mesh Estimationcited by 81
  25. 2022P-STMO: Pre-Trained Spatial Temporal Many-to-One Model for 3D Human Pose Estimationcited by 80
  26. 20163D Human Pose Estimation from a Single Image via Distance Matrix Regressioncited by 73
  27. 2017Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Imagecited by 73
  28. 2020FrankMocap: Fast Monocular 3D Hand and Body Motion Capture by Regression and Integrationcited by 72
  29. 2023AssemblyHands: Towards Egocentric Activity Understanding via 3D Hand Pose Estimationcited by 64
  30. 2025HaWoR: World-Space Hand Motion Reconstruction from Egocentric Videoscited by 62
  31. 2024DiffH2O: Diffusion-Based Synthesis of Hand-Object Interactions from Textual Descriptionscited by 60
  32. 2024EM Distillation for One-step Diffusion Modelscited by 60
  33. 2024Estimating Body and Hand Motion in an Ego-sensed Worldcited by 56
  34. 2024Dyn-HaMR: Recovering 4D Interacting Hand Motion from a Dynamic Cameracited by 50
  35. 2024Introducing HOT3D: An Egocentric Dataset for 3D Hand and Object Trackingcited by 46
  36. 2023MAS: Multi-view Ancestral Sampling for 3D motion generation using 2D diffusioncited by 37
  37. 2021ElePose: Unsupervised 3D Human Pose Estimation by Predicting Camera Elevation and Learning Normalizing Flows on 2D Posescited by 35
  38. 2022Hierarchical Temporal Transformer for 3D Hand Pose Estimation and Action Recognition from Egocentric RGB Videoscited by 35

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026HandX: Scaling Bimanual Motion and Interaction Generationcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025Text-Driven 3D Hand Motion Generation from Sign Language Data
  2. 2025Novel Diffusion Models for Multimodal 3D Hand Trajectory Prediction
  3. 2023Denoising Diffusion for 3D Hand Pose Estimation from Images
  4. 2024HOIDiffusion: Generating Realistic 3D Hand-Object Interaction Data
  5. 2026CLUTCH: Contextualized Language model for Unlocking Text-Conditioned Hand motion modelling in the wild
  6. 2026Teacher-Student Diffusion Model for Text-Driven 3D Hand Motion Generation
  7. 2023BOTH2Hands: Inferring 3D Hands from Both Text Prompts and Body Dynamics
  8. 2026EggHand: A Multimodal Foundation Model for Egocentric Hand Pose Forecasting