# Procrustean Regression Networks: Learning 3D Structure of Non-Rigid Objects from 2D Annotations

arXiv: https://arxiv.org/abs/2007.10961 · HTML: https://mlsanity.com/paper/2007.10961/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jul 2020.

## Built on by (most cited first, 13 of 13)

- [State of the Art in Dense Monocular Non-Rigid 3D Reconstruction](https://mlsanity.com/paper/2210.15664/) (2022)
- [PAUL: Procrustean Autoencoder for Unsupervised Lifting](https://mlsanity.com/paper/2103.16773/) (2021)
- [PR-RRN: Pairwise-Regularized Residual-Recursive Networks for Non-rigid Structure-from-Motion](https://mlsanity.com/paper/2108.07506/) (2021)
- [Deep Non-rigid Structure-from-Motion: A Sequence-to-Sequence Translation Perspective](https://mlsanity.com/paper/2204.04730/) (2022)
- [MHR-Net: Multiple-Hypothesis Reconstruction of Non-Rigid Shapes from 2D Views](https://mlsanity.com/paper/2207.09086/) (2022)
- [Unsupervised 3D Pose Estimation with Non-Rigid Structure-from-Motion Modeling](https://mlsanity.com/paper/2308.10705/) (2023)
- [A Kendall Shape Space Approach to 3D Shape Estimation from 2D Landmarks](https://mlsanity.com/paper/2207.12687/) (2022)
- [High Fidelity 3D Reconstructions with Limited Physical Views](https://mlsanity.com/paper/2110.11599/) (2021)
- [End-to-End Learning of Multi-category 3D Pose and Shape Estimation](https://mlsanity.com/paper/2112.10196/) (2021)
- [Piecewise Planar Hulls for Semi-Supervised Learning of 3D Shape and Pose from 2D Images](https://mlsanity.com/paper/2211.07491/) (2022)
- [Non-rigid Structure-from-Motion: Temporally-smooth Procrustean Alignment and Spatially-variant Deformation Modeling](https://mlsanity.com/paper/2405.04309/) (2024)
- [Challenges in Non-Polymeric Crystal Structure Prediction: Why a Geometric, Permutation-Invariant Loss is Needed](https://mlsanity.com/paper/2509.00832/) (2025)
- [SSR: A Training-Free Approach for Streaming 3D Reconstruction](https://mlsanity.com/paper/2603.14765/) (2026)

## Abstract (arXiv)

We propose a novel framework for training neural networks which is capable of learning 3D information of non-rigid objects when only 2D annotations are available as ground truths. Recently, there have been some approaches that incorporate the problem setting of non-rigid structure-from-motion (NRSfM) into deep learning to learn 3D structure reconstruction. The most important difficulty of NRSfM is to estimate both the rotation and deformation at the same time, and previous works handle this by regressing both of them. In this paper, we resolve this difficulty by proposing a loss function wherein the suitable rotation is automatically determined. Trained with the cost function consisting of the reprojection error and the low-rank term of aligned shapes, the network learns the 3D structures of such objects as human skeletons and faces during the training, whereas the testing is done in a single-frame basis. The proposed method can handle inputs with missing entries and experimental results validate that the proposed framework shows superior reconstruction performance to the state-of-the-art method on the Human 3.6M, 300-VW, and SURREAL datasets, even though the underlying network structure is very simple.