# HandFlow: Quantifying View-Dependent 3D Ambiguity in Two-Hand Reconstruction with Normalizing Flow

arXiv: https://arxiv.org/abs/2210.01692 · HTML: https://mlsanity.com/paper/2210.01692/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 39 references that are in the map.
- First posted on arXiv 4 Oct 2022.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Learning joint reconstruction of hands and manipulated objects](https://mlsanity.com/paper/1904.05767/) (2019)
- [InterHand2.6M: A Dataset and Baseline for 3D Interacting Hand Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/2008.09309/) (2020)
- [Hand Keypoint Detection in Single Images using Multiview Bootstrapping](https://mlsanity.com/paper/1704.07809/) (2017)
- [3D Hand Shape and Pose from Images in the Wild](https://mlsanity.com/paper/1902.03451/) (2019)
- [Learning Likelihoods with Conditional Normalizing Flows](https://mlsanity.com/paper/1912.00042/) (2019)
- [Probabilistic Modeling for Human Mesh Recovery](https://mlsanity.com/paper/2108.11944/) (2021)
- [GANerated Hands for Real-time 3D Hand Tracking from Monocular RGB](https://mlsanity.com/paper/1712.01057/) (2017)
- [Leveraging Photometric Consistency over Time for Sparsely Supervised Hand-Object Reconstruction](https://mlsanity.com/paper/2004.13449/) (2020)
- [Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/1907.11346/) (2019)
- [Capturing Hands in Action using Discriminative Salient Points and Physics Simulation](https://mlsanity.com/paper/1506.02178/) (2015)
- [Weakly Supervised 3D Human Pose and Shape Reconstruction with Normalizing Flows](https://mlsanity.com/paper/2003.10350/) (2020)
- [Interacting Attention Graph for Single Image Two-Hand Reconstruction](https://mlsanity.com/paper/2203.09364/) (2022)
- [Probabilistic Monocular 3D Human Pose Estimation with Normalizing Flows](https://mlsanity.com/paper/2107.13788/) (2021)
- [Real-time Pose and Shape Reconstruction of Two Interacting Hands With a Single Depth Camera](https://mlsanity.com/paper/2106.08059/) (2021)
- [Learning to Disambiguate Strongly Interacting Hands via Probabilistic Per-pixel Part Segmentation](https://mlsanity.com/paper/2107.00434/) (2021)
- [Fast and Robust Hand Tracking Using Detection-Guided Optimization](https://mlsanity.com/paper/1602.04124/) (2016)
- [Two-hand Global 3D Pose Estimation Using Monocular RGB](https://mlsanity.com/paper/2006.01320/) (2020)
- [HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation from a Single Depth Map](https://mlsanity.com/paper/2004.01588/) (2020)
- [Occlusion-aware Hand Pose Estimation Using Hierarchical Mixture Density Network](https://mlsanity.com/paper/1711.10872/) (2017)
- [Informed MCMC with Bayesian Neural Networks for Facial Image Analysis](https://mlsanity.com/paper/1811.07969/) (2018)

## Built on by (most cited first, 5 of 5)

- [State of the Art in Dense Monocular Non-Rigid 3D Reconstruction](https://mlsanity.com/paper/2210.15664/) (2022)
- [Decaf: Monocular Deformation Capture for Face and Hand Interactions](https://mlsanity.com/paper/2309.16670/) (2023)
- [3D Pose Estimation of Two Interacting Hands from a Monocular Event Camera](https://mlsanity.com/paper/2312.14157/) (2023)
- [StackFLOW: Monocular Human-Object Reconstruction by Stacked Normalizing Flow with Offset](https://mlsanity.com/paper/2407.20545/) (2024)
- [Learning Correlation-aware Aleatoric Uncertainty for 3D Hand Pose Estimation](https://mlsanity.com/paper/2509.01242/) (2025)

## Abstract (arXiv)

Reconstructing two-hand interactions from a single image is a challenging problem due to ambiguities that stem from projective geometry and heavy occlusions. Existing methods are designed to estimate only a single pose, despite the fact that there exist other valid reconstructions that fit the image evidence equally well. In this paper we propose to address this issue by explicitly modeling the distribution of plausible reconstructions in a conditional normalizing flow framework. This allows us to directly supervise the posterior distribution through a novel determinant magnitude regularization, which is key to varied 3D hand pose samples that project well into the input image. We also demonstrate that metrics commonly used to assess reconstruction quality are insufficient to evaluate pose predictions under such severe ambiguity. To address this, we release the first dataset with multiple plausible annotations per image called MultiHands. The additional annotations enable us to evaluate the estimated distribution using the maximum mean discrepancy metric. Through this, we demonstrate the quality of our probabilistic reconstruction and show that explicit ambiguity modeling is better-suited for this challenging problem.