# Jumping Manifolds: Geometry Aware Dense Non-Rigid Structure from Motion

arXiv: https://arxiv.org/abs/1902.01077 · HTML: https://mlsanity.com/paper/1902.01077/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Feb 2019.

## Built on by (most cited first, 12 of 12)

- [State of the Art in Dense Monocular Non-Rigid 3D Reconstruction](https://mlsanity.com/paper/2210.15664/) (2022)
- [Neural Radiance Fields Approach to Deep Multi-View Photometric Stereo](https://mlsanity.com/paper/2110.05594/) (2021)
- [Organic Priors in Non-Rigid Structure from Motion](https://mlsanity.com/paper/2207.06262/) (2022)
- [PR-RRN: Pairwise-Regularized Residual-Recursive Networks for Non-rigid Structure-from-Motion](https://mlsanity.com/paper/2108.07506/) (2021)
- [A geometry-aware deep network for depth estimation in monocular endoscopy](https://mlsanity.com/paper/2304.10241/) (2023)
- [Neural Architecture Search for Efficient Uncalibrated Deep Photometric Stereo](https://mlsanity.com/paper/2110.05621/) (2021)
- [MHR-Net: Multiple-Hypothesis Reconstruction of Non-Rigid Shapes from 2D Views](https://mlsanity.com/paper/2207.09086/) (2022)
- [DeepPS2: Revisiting Photometric Stereo Using Two Differently Illuminated Images](https://mlsanity.com/paper/2207.02025/) (2022)
- [Thin-Shell-SfT: Fine-Grained Monocular Non-rigid 3D Surface Tracking with Neural Deformation Fields](https://mlsanity.com/paper/2503.19976/) (2025)
- [Non-Rigid Structure-from-Motion via Differential Geometry with Recoverable Conformal Scale](https://mlsanity.com/paper/2510.01665/) (2025)
- [Color Image Set Recognition Based on Quaternionic Grassmannians](https://mlsanity.com/paper/2505.23629/) (2025)
- [Rethinking Dense Optical Flow without Test-Time Scaling](https://mlsanity.com/paper/2605.08000/) (2026)

## Abstract (arXiv)

Given dense image feature correspondences of a non-rigidly moving object across multiple frames, this paper proposes an algorithm to estimate its 3D shape for each frame. To solve this problem accurately, the recent state-of-the-art algorithm reduces this task to set of local linear subspace reconstruction and clustering problem using Grassmann manifold representation citekumar2018scalable. Unfortunately, their method missed on some of the critical issues associated with the modeling of surface deformations, for e.g., the dependence of a local surface deformation on its neighbors. Furthermore, their representation to group high dimensional data points inevitably introduce the drawbacks of categorizing samples on the high-dimensional Grassmann manifold citehuang2015projection, harandi2014manifold. Hence, to deal with such limitations with citekumar2018scalable, we propose an algorithm that jointly exploits the benefit of high-dimensional Grassmann manifold to perform reconstruction, and its equivalent lower-dimensional representation to infer suitable clusters. To accomplish this, we project each Grassmannians onto a lower-dimensional Grassmann manifold which preserves and respects the deformation of the structure w.r.t its neighbors. These Grassmann points in the lower-dimension then act as a representative for the selection of high-dimensional Grassmann samples to perform each local reconstruction. In practice, our algorithm provides a geometrically efficient way to solve dense NRSfM by switching between manifolds based on its benefit and usage. Experimental results show that the proposed algorithm is very effective in handling noise with reconstruction accuracy as good as or better than the competing methods.