# HiFiHR: Enhancing 3D Hand Reconstruction from a Single Image via High-Fidelity Texture

arXiv: https://arxiv.org/abs/2308.13628 · HTML: https://mlsanity.com/paper/2308.13628/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 41 references that are in the map.
- First posted on arXiv 25 Aug 2023.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video](https://mlsanity.com/paper/2201.04127/) (2022)
- [Learning joint reconstruction of hands and manipulated objects](https://mlsanity.com/paper/1904.05767/) (2019)
- [FreiHAND: A Dataset for Markerless Capture of Hand Pose and Shape from Single RGB Images](https://mlsanity.com/paper/1909.04349/) (2019)
- [NeuMan: Neural Human Radiance Field from a Single Video](https://mlsanity.com/paper/2203.12575/) (2022)
- [Learning to Estimate 3D Hand Pose from Single RGB Images](https://mlsanity.com/paper/1705.01389/) (2017)
- [A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose](https://mlsanity.com/paper/2102.06199/) (2021)
- [Hand Keypoint Detection in Single Images using Multiview Bootstrapping](https://mlsanity.com/paper/1704.07809/) (2017)
- [HeadNeRF: A Real-time NeRF-based Parametric Head Model](https://mlsanity.com/paper/2112.05637/) (2021)
- [Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer](https://mlsanity.com/paper/1908.01210/) (2019)
- [3D Hand Shape and Pose from Images in the Wild](https://mlsanity.com/paper/1902.03451/) (2019)
- [H-NeRF: Neural Radiance Fields for Rendering and Temporal Reconstruction of Humans in Motion](https://mlsanity.com/paper/2110.13746/) (2021)
- [Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstruction](https://mlsanity.com/paper/2004.13449/) (2020)
- [Weakly Supervised 3D Hand Pose Estimation via Biomechanical Constraints](https://mlsanity.com/paper/2003.09282/) (2020)
- [Model-based 3D Hand Reconstruction via Self-Supervised Learning](https://mlsanity.com/paper/2103.11703/) (2021)
- [LISA: Learning Implicit Shape and Appearance of Hands](https://mlsanity.com/paper/2204.01695/) (2022)
- [DeepHandMesh: A Weakly-supervised Deep Encoder-Decoder Framework for High-fidelity Hand Mesh Modeling](https://mlsanity.com/paper/2008.08213/) (2020)
- [Surface-Aligned Neural Radiance Fields for Controllable 3D Human Synthesis](https://mlsanity.com/paper/2201.01683/) (2022)
- [Hand Avatar: Free-Pose Hand Animation and Rendering from Monocular Video](https://mlsanity.com/paper/2211.12782/) (2022)
- [Hand Image Understanding via Deep Multi-Task Learning](https://mlsanity.com/paper/2107.11646/) (2021)
- [HARP: Personalized Hand Reconstruction from a Monocular RGB Video](https://mlsanity.com/paper/2212.09530/) (2022)
- [LiveHand: Real-time and Photorealistic Neural Hand Rendering](https://mlsanity.com/paper/2302.07672/) (2023)
- [DART: Articulated Hand Model with Diverse Accessories and Rich Textures](https://mlsanity.com/paper/2210.07650/) (2022)
- [End-to-end Weakly-supervised Single-stage Multiple 3D Hand Mesh Reconstruction from a Single RGB Image](https://mlsanity.com/paper/2204.08154/) (2022)

## Built on by (most cited first, 2 of 2)

- [GenHeld: Generating and Editing Handheld Objects](https://mlsanity.com/paper/2406.05059/) (2024)
- [Enhancing Monocular 3D Hand Reconstruction with Learned Texture Priors](https://mlsanity.com/paper/2508.09629/) (2025)

## Abstract (arXiv)

We present HiFiHR, a high-fidelity hand reconstruction approach that utilizes render-and-compare in the learning-based framework from a single image, capable of generating visually plausible and accurate 3D hand meshes while recovering realistic textures. Our method achieves superior texture reconstruction by employing a parametric hand model with predefined texture assets, and by establishing a texture reconstruction consistency between the rendered and input images during training. Moreover, based on pretraining the network on an annotated dataset, we apply varying degrees of supervision using our pipeline, i.e., self-supervision, weak supervision, and full supervision, and discuss the various levels of contributions of the learned high-fidelity textures in enhancing hand pose and shape estimation. Experimental results on public benchmarks including FreiHAND and HO-3D demonstrate that our method outperforms the state-of-the-art hand reconstruction methods in texture reconstruction quality while maintaining comparable accuracy in pose and shape estimation. Our code is available at https://github.com/viridityzhu/HiFiHR.