# MHR-Net: Multiple-Hypothesis Reconstruction of Non-Rigid Shapes from 2D Views

arXiv: https://arxiv.org/abs/2207.09086 · HTML: https://mlsanity.com/paper/2207.09086/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 70 references that are in the map.
- First posted on arXiv 19 Jul 2022.

## Builds on (most cited first, 21 of 21)

- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [3D human pose estimation in video with temporal convolutions and semi-supervised training](https://mlsanity.com/paper/1811.11742/) (2018)
- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [Learning from Synthetic Humans](https://mlsanity.com/paper/1701.01370/) (2017)
- [Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypotheses](https://mlsanity.com/paper/1612.00197/) (2016)
- [Generating Multiple Hypotheses for 3D Human Pose Estimation with Mixture Density Network](https://mlsanity.com/paper/1904.05547/) (2019)
- [Probabilistic Monocular 3D Human Pose Estimation with Normalizing Flows](https://mlsanity.com/paper/2107.13788/) (2021)
- [Stochastic Multiple Choice Learning for Training Diverse Deep Ensembles](https://mlsanity.com/paper/1606.07839/) (2016)
- [Monocular 3D Human Pose Estimation by Generation and Ordinal Ranking](https://mlsanity.com/paper/1904.01324/) (2019)
- [C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion](https://mlsanity.com/paper/1909.02533/) (2019)
- [Weakly Supervised Generative Network for Multiple 3D Human Pose Hypotheses](https://mlsanity.com/paper/2008.05770/) (2020)
- [Scalable Dense Non-rigid Structure-from-Motion: A Grassmannian Perspective](https://mlsanity.com/paper/1803.00233/) (2018)
- [Unsupervised Adversarial Learning of 3D Human Pose from 2D Joint Locations](https://mlsanity.com/paper/1803.08244/) (2018)
- [Distill Knowledge from NRSfM for Weakly Supervised 3D Pose Learning](https://mlsanity.com/paper/1908.06377/) (2019)
- [Procrustean Regression Networks: Learning 3D Structure of Non-Rigid Objects from 2D Annotations](https://mlsanity.com/paper/2007.10961/) (2020)
- [Jumping Manifolds: Geometry Aware Dense Non-Rigid Structure from Motion](https://mlsanity.com/paper/1902.01077/) (2019)
- [Invariant Teacher and Equivariant Student for Unsupervised 3D Human Pose Estimation](https://mlsanity.com/paper/2012.09398/) (2020)
- [PAUL: Procrustean Autoencoder for Unsupervised Lifting](https://mlsanity.com/paper/2103.16773/) (2021)
- [PR-RRN: Pairwise-Regularized Residual-Recursive Networks for Non-rigid Structure-from-Motion](https://mlsanity.com/paper/2108.07506/) (2021)
- [Accurate Optimization of Weighted Nuclear Norm for Non-Rigid Structure from Motion](https://mlsanity.com/paper/2003.10281/) (2020)
- [Incremental Non-Rigid Structure-from-Motion with Unknown Focal Length](https://mlsanity.com/paper/1808.04181/) (2018)

## Built on by (most cited first, 7 of 7)

- [Recent Trends in 3D Reconstruction of General Non-Rigid Scenes](https://mlsanity.com/paper/2403.15064/) (2024)
- [3D-LFM: Lifting Foundation Model](https://mlsanity.com/paper/2312.11894/) (2023)
- [Unsupervised 3D Pose Estimation with Non-Rigid Structure-from-Motion Modeling](https://mlsanity.com/paper/2308.10705/) (2023)
- [PADS: Plug-and-Play 3D Human Pose Analysis via Diffusion Generative Modeling](https://mlsanity.com/paper/2401.08930/) (2024)
- [Non-rigid Structure-from-Motion: Temporally-smooth Procrustean Alignment and Spatially-variant Deformation Modeling](https://mlsanity.com/paper/2405.04309/) (2024)
- [Unsupervised 2D-3D lifting of non-rigid objects using local constraints](https://mlsanity.com/paper/2504.19227/) (2025)
- [SSR: A Training-Free Approach for Streaming 3D Reconstruction](https://mlsanity.com/paper/2603.14765/) (2026)

## Abstract (arXiv)

We propose MHR-Net, a novel method for recovering Non-Rigid Shapes from Motion (NRSfM). MHR-Net aims to find a set of reasonable reconstructions for a 2D view, and it also selects the most likely reconstruction from the set. To deal with the challenging unsupervised generation of non-rigid shapes, we develop a new Deterministic Basis and Stochastic Deformation scheme in MHR-Net. The non-rigid shape is first expressed as the sum of a coarse shape basis and a flexible shape deformation, then multiple hypotheses are generated with uncertainty modeling of the deformation part. MHR-Net is optimized with reprojection loss on the basis and the best hypothesis. Furthermore, we design a new Procrustean Residual Loss, which reduces the rigid rotations between similar shapes and further improves the performance. Experiments show that MHR-Net achieves state-of-the-art reconstruction accuracy on Human3.6M, SURREAL and 300-VW datasets.