# ScaleHP: Scale-Mediated Optimization of Coupled Errors for Metric-Space Hand Pose Estimation

arXiv: https://arxiv.org/abs/2606.25619 · HTML: https://mlsanity.com/paper/2606.25619/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 50 references that are in the map.
- First posted on arXiv 24 Jun 2026.

## Builds on (most cited first, 30 of 30)

- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [End-to-End Human Pose and Mesh Reconstruction with Transformers](https://mlsanity.com/paper/2012.09760/) (2020)
- [Learning joint reconstruction of hands and manipulated objects](https://mlsanity.com/paper/1904.05767/) (2019)
- [Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Image](https://mlsanity.com/paper/2307.10984/) (2023)
- [Reconstructing Hands in 3D with Transformers](https://mlsanity.com/paper/2312.05251/) (2023)
- [I2L-MeshNet: Image-to-Lixel Prediction Network for Accurate 3D Human Pose and Mesh Estimation from a Single RGB Image](https://mlsanity.com/paper/2008.03713/) (2020)
- [FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Images](https://mlsanity.com/paper/1909.04349/) (2019)
- [MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Details](https://mlsanity.com/paper/2507.02546/) (2025)
- [3D Hand Shape and Pose from Images in the Wild](https://mlsanity.com/paper/1902.03451/) (2019)
- [3D Hand Shape and Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/1903.00812/) (2019)
- [Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild](https://mlsanity.com/paper/2004.01946/) (2020)
- [Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstruction](https://mlsanity.com/paper/2004.13449/) (2020)
- [Hand Pose Estimation via Latent 2.5D Heatmap Regression](https://mlsanity.com/paper/1804.09534/) (2018)
- [Weakly Supervised 3D Hand Pose Estimation via Biomechanical Constraints](https://mlsanity.com/paper/2003.09282/) (2020)
- [HandOccNet: Occlusion-Robust 3D Hand Mesh Estimation Network](https://mlsanity.com/paper/2203.14564/) (2022)
- [ArtiBoost: Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesis](https://mlsanity.com/paper/2109.05488/) (2021)
- [MobRecon: Mobile-Friendly Hand Mesh Reconstruction from Monocular Image](https://mlsanity.com/paper/2112.02753/) (2021)
- [Camera-Space Hand Mesh Recovery via Semantic Aggregation and Adaptive 2D-1D Registration](https://mlsanity.com/paper/2103.02845/) (2021)
- [Hamba: Single-view 3D Hand Reconstruction with Graph-guided Bi-Scanning Mamba](https://mlsanity.com/paper/2407.09646/) (2024)
- [AiOS: All-in-One-Stage Expressive Human Pose and Shape Estimation](https://mlsanity.com/paper/2403.17934/) (2024)
- [A Simple Baseline for Efficient Hand Mesh Reconstruction](https://mlsanity.com/paper/2403.01813/) (2024)
- [ScaleDepth: Decomposing Metric Depth Estimation into Scale Prediction and Relative Depth Estimation](https://mlsanity.com/paper/2407.08187/) (2024)
- [HandOS: 3D Hand Reconstruction in One Stage](https://mlsanity.com/paper/2412.01537/) (2024)
- [HandDGP: Camera-Space Hand Mesh Prediction with Differentiable Global Positioning](https://mlsanity.com/paper/2407.15844/) (2024)
- [MaskHand: Generative Masked Modeling for Robust Hand Mesh Reconstruction in the Wild](https://mlsanity.com/paper/2412.13393/) (2024)
- [HandGCAT: Occlusion-Robust 3D Hand Mesh Reconstruction from Monocular Images](https://mlsanity.com/paper/2403.07912/) (2024)
- [MetricHMSR:Metric Human Mesh and Scene Recovery from Monocular Images](https://mlsanity.com/paper/2506.09919/) (2025)
- [EgoForce: Forearm-Guided Camera-Space 3D Hand Pose from a Monocular Egocentric Camera](https://mlsanity.com/paper/2605.12498/) (2026)

## Abstract (arXiv)

In this paper, we present ScaleHP, a unified framework that explicitly represents per-instance metric scale to resolve the coupled errors in calibrated camera-space hand pose estimation. Under the common root-relative-to-global paradigm, camera-space accuracy depends jointly on relative geometry, root localization, and their scale-dependent composition. ScaleHP treats scale as the shared interface among these terms rather than optimizing them in isolation. Its metric-aware 2D-3D Transformer decoder introduces a dedicated scale token that exchanges information with sparse, semantically indexed 2D/3D joint queries, allowing global metric reasoning and local joint geometry to mutually refine one another. A parameter-free calibrated module then recovers root translation from the predicted unit-scale joints, metric scale, and known camera intrinsics. ScaleHP demonstrates state-of-the-art CS-MPJPE on FreiHand (35.8 mm) and, under benchmark-specific training protocols, state-of-the-art PA-MPJPE on FreiHand, DexYCB, and HO3Dv3 (5.0/4.6/5.9 mm), showing that improved global metric localization is attained together with improved local articulation. Ablations further show that scale-pose interaction improves both camera-space accuracy and relative depth geometry.