# PR-RRN: Pairwise-Regularized Residual-Recursive Networks for Non-rigid Structure-from-Motion

arXiv: https://arxiv.org/abs/2108.07506 · HTML: https://mlsanity.com/paper/2108.07506/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 52 references that are in the map.
- First posted on arXiv 17 Aug 2021.

## Builds on (most cited first, 15 of 15)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Unsupervised 3D Pose Estimation with Geometric Self-Supervision](https://mlsanity.com/paper/1904.04812/) (2019)
- [An Analysis of SVD for Deep Rotation Estimation](https://mlsanity.com/paper/2006.14616/) (2020)
- [Info3D: Representation Learning on 3D Objects using Mutual Information Maximization and Contrastive Learning](https://mlsanity.com/paper/2006.02598/) (2020)
- [C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion](https://mlsanity.com/paper/1909.02533/) (2019)
- [Pixel Recursive Super Resolution](https://mlsanity.com/paper/1702.00783/) (2017)
- [Scalable Dense Non-rigid Structure-from-Motion: A Grassmannian Perspective](https://mlsanity.com/paper/1803.00233/) (2018)
- [Structure from Recurrent Motion: From Rigidity to Recurrency](https://mlsanity.com/paper/1804.06510/) (2018)
- [Self-Supervised Visibility Learning for Novel View Synthesis](https://mlsanity.com/paper/2103.15407/) (2021)
- [Procrustean Regression Networks: Learning 3D Structure of Non-Rigid Objects from 2D Annotations](https://mlsanity.com/paper/2007.10961/) (2020)
- [Jumping Manifolds: Geometry Aware Dense Non-Rigid Structure from Motion](https://mlsanity.com/paper/1902.01077/) (2019)
- [Spatial-Temporal Union of Subspaces for Multi-body Non-rigid Structure-from-Motion](https://mlsanity.com/paper/1705.04916/) (2017)

## Built on by (most cited first, 9 of 9)

- [State of the Art in Dense Monocular Non-Rigid 3D Reconstruction](https://mlsanity.com/paper/2210.15664/) (2022)
- [Recent Trends in 3D Reconstruction of General Non-Rigid Scenes](https://mlsanity.com/paper/2403.15064/) (2024)
- [CAMM: Building Category-Agnostic and Animatable 3D Models from Monocular Videos](https://mlsanity.com/paper/2304.06937/) (2023)
- [Deep Non-rigid Structure-from-Motion: A Sequence-to-Sequence Translation Perspective](https://mlsanity.com/paper/2204.04730/) (2022)
- [MHR-Net: Multiple-Hypothesis Reconstruction of Non-Rigid Shapes from 2D Views](https://mlsanity.com/paper/2207.09086/) (2022)
- [Unsupervised 3D Pose Estimation with Non-Rigid Structure-from-Motion Modeling](https://mlsanity.com/paper/2308.10705/) (2023)
- [End-to-End Learning of Multi-category 3D Pose and Shape Estimation](https://mlsanity.com/paper/2112.10196/) (2021)
- [Piecewise Planar Hulls for Semi-Supervised Learning of 3D Shape and Pose from 2D Images](https://mlsanity.com/paper/2211.07491/) (2022)
- [SSR: A Training-Free Approach for Streaming 3D Reconstruction](https://mlsanity.com/paper/2603.14765/) (2026)

## Abstract (arXiv)

We propose PR-RRN, a novel neural-network based method for Non-rigid Structure-from-Motion (NRSfM). PR-RRN consists of Residual-Recursive Networks (RRN) and two extra regularization losses. RRN is designed to effectively recover 3D shape and camera from 2D keypoints with novel residual-recursive structure. As NRSfM is a highly under-constrained problem, we propose two new pairwise regularization to further regularize the reconstruction. The Rigidity-based Pairwise Contrastive Loss regularizes the shape representation by encouraging higher similarity between the representations of high-rigidity pairs of frames than low-rigidity pairs. We propose minimum singular-value ratio to measure the pairwise rigidity. The Pairwise Consistency Loss enforces the reconstruction to be consistent when the estimated shapes and cameras are exchanged between pairs. Our approach achieves state-of-the-art performance on CMU MOCAP and PASCAL3D+ dataset.