# Non-rigid Structure-from-Motion: Temporally-smooth Procrustean Alignment and Spatially-variant Deformation Modeling

arXiv: https://arxiv.org/abs/2405.04309 · HTML: https://mlsanity.com/paper/2405.04309/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 51 references that are in the map.
- First posted on arXiv 7 May 2024.

## Builds on (most cited first, 12 of 12)

- [C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion](https://mlsanity.com/paper/1909.02533/) (2019)
- [H3WB: Human3.6M 3D WholeBody Dataset and Benchmark](https://mlsanity.com/paper/2211.15692/) (2022)
- [Scalable Dense Non-rigid Structure-from-Motion: A Grassmannian Perspective](https://mlsanity.com/paper/1803.00233/) (2018)
- [Non-Rigid Structure from Motion: Prior-Free Factorization Method Revisited](https://mlsanity.com/paper/1902.10274/) (2019)
- [Organic Priors in Non-Rigid Structure from Motion](https://mlsanity.com/paper/2207.06262/) (2022)
- [A Benchmark and Evaluation of Non-Rigid Structure from Motion](https://mlsanity.com/paper/1801.08388/) (2018)
- [Procrustean Regression Networks: Learning 3D Structure of Non-Rigid Objects from 2D Annotations](https://mlsanity.com/paper/2007.10961/) (2020)
- [Deep Non-rigid Structure-from-Motion: A Sequence-to-Sequence Translation Perspective](https://mlsanity.com/paper/2204.04730/) (2022)
- [Robust Isometric Non-Rigid Structure-from-Motion](https://mlsanity.com/paper/2010.04690/) (2020)
- [MHR-Net: Multiple-Hypothesis Reconstruction of Non-Rigid Shapes from 2D Views](https://mlsanity.com/paper/2207.09086/) (2022)
- [Unsupervised 3D Pose Estimation with Non-Rigid Structure-from-Motion Modeling](https://mlsanity.com/paper/2308.10705/) (2023)
- [Accurate Optimization of Weighted Nuclear Norm for Non-Rigid Structure from Motion](https://mlsanity.com/paper/2003.10281/) (2020)

## Abstract (arXiv)

Even though Non-rigid Structure-from-Motion (NRSfM) has been extensively studied and great progress has been made, there are still key challenges that hinder their broad real-world applications: 1) the inherent motion/rotation ambiguity requires either explicit camera motion recovery with extra constraint or complex Procrustean Alignment; 2) existing low-rank modeling of the global shape can over-penalize drastic deformations in the 3D shape sequence. This paper proposes to resolve the above issues from a spatial-temporal modeling perspective. First, we propose a novel Temporally-smooth Procrustean Alignment module that estimates 3D deforming shapes and adjusts the camera motion by aligning the 3D shape sequence consecutively. Our new alignment module remedies the requirement of complex reference 3D shape during alignment, which is more conductive to non-isotropic deformation modeling. Second, we propose a spatial-weighted approach to enforce the low-rank constraint adaptively at different locations to accommodate drastic spatially-variant deformation reconstruction better. Our modeling outperform existing low-rank based methods, and extensive experiments across different datasets validate the effectiveness of our method.