# PAUL: Procrustean Autoencoder for Unsupervised Lifting

arXiv: https://arxiv.org/abs/2103.16773 · HTML: https://mlsanity.com/paper/2103.16773/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 44 references that are in the map.
- First posted on arXiv 31 Mar 2021.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [Learning from Synthetic Humans](https://mlsanity.com/paper/1701.01370/) (2017)
- [Unite the People: Closing the Loop Between 3D and 2D Human Representations](https://mlsanity.com/paper/1701.02468/) (2017)
- [From Variational to Deterministic Autoencoders](https://mlsanity.com/paper/1903.12436/) (2019)
- [Unsupervised 3D Pose Estimation with Geometric Self-Supervision](https://mlsanity.com/paper/1904.04812/) (2019)
- [RepNet: Weakly Supervised Training of an Adversarial Reprojection Network for 3D Human Pose Estimation](https://mlsanity.com/paper/1902.09868/) (2019)
- [C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion](https://mlsanity.com/paper/1909.02533/) (2019)
- [Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video](https://mlsanity.com/paper/1511.09439/) (2015)
- [Can 3D Pose be Learned from 2D Projections Alone?](https://mlsanity.com/paper/1808.07182/) (2018)
- [Deep Non-Rigid Structure from Motion](https://mlsanity.com/paper/1908.00052/) (2019)
- [Unsupervised Adversarial Learning of 3D Human Pose from 2D Joint Locations](https://mlsanity.com/paper/1803.08244/) (2018)
- [Non-Rigid Structure from Motion: Prior-Free Factorization Method Revisited](https://mlsanity.com/paper/1902.10274/) (2019)
- [Multi-body Non-rigid Structure-from-Motion](https://mlsanity.com/paper/1607.04515/) (2016)
- [Procrustean Regression Networks: Learning 3D Structure of Non-Rigid Objects from 2D Annotations](https://mlsanity.com/paper/2007.10961/) (2020)
- [Variational Auto-Decoder: A Method for Neural Generative Modeling from Incomplete Data](https://mlsanity.com/paper/1903.00840/) (2019)
- [Deep NRSfM++: Towards Unsupervised 2D-3D Lifting in the Wild](https://mlsanity.com/paper/2001.10090/) (2020)

## Built on by (most cited first, 9 of 9)

- [State of the Art in Dense Monocular Non-Rigid 3D Reconstruction](https://mlsanity.com/paper/2210.15664/) (2022)
- [Unbiased 4D: Monocular 4D Reconstruction with a Neural Deformation Model](https://mlsanity.com/paper/2206.08368/) (2022)
- [Deep Non-rigid Structure-from-Motion: A Sequence-to-Sequence Translation Perspective](https://mlsanity.com/paper/2204.04730/) (2022)
- [MHR-Net: Multiple-Hypothesis Reconstruction of Non-Rigid Shapes from 2D Views](https://mlsanity.com/paper/2207.09086/) (2022)
- [Unsupervised 3D Pose Estimation with Non-Rigid Structure-from-Motion Modeling](https://mlsanity.com/paper/2308.10705/) (2023)
- [End-to-End Learning of Multi-category 3D Pose and Shape Estimation](https://mlsanity.com/paper/2112.10196/) (2021)
- [Non-Rigid Structure-from-Motion via Differential Geometry with Recoverable Conformal Scale](https://mlsanity.com/paper/2510.01665/) (2025)
- [Piecewise Planar Hulls for Semi-Supervised Learning of 3D Shape and Pose from 2D Images](https://mlsanity.com/paper/2211.07491/) (2022)
- [Object Agnostic 3D Lifting in Space and Time](https://mlsanity.com/paper/2412.01166/) (2024)

## Abstract (arXiv)

Recent success in casting Non-rigid Structure from Motion (NRSfM) as an unsupervised deep learning problem has raised fundamental questions about what novelty in NRSfM prior could the deep learning offer. In this paper we advocate for a 3D deep auto-encoder framework to be used explicitly as the NRSfM prior. The framework is unique as: (i) it learns the 3D auto-encoder weights solely from 2D projected measurements, and (ii) it is Procrustean in that it jointly resolves the unknown rigid pose for each shape instance. We refer to this architecture as a Procustean Autoencoder for Unsupervised Lifting (PAUL), and demonstrate state-of-the-art performance across a number of benchmarks in comparison to recent innovations such as Deep NRSfM and C3PDO.