# Image-Guided Shape-from-Template Using Mesh Inextensibility Constraints

arXiv: https://arxiv.org/abs/2507.22699 · HTML: https://mlsanity.com/paper/2507.22699/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 57 references that are in the map.
- First posted on arXiv 30 Jul 2025.

## Builds on (most cited first, 21 of 21)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction](https://mlsanity.com/paper/2309.13101/) (2023)
- [Modular Primitives for High-Performance Differentiable Rendering](https://mlsanity.com/paper/2011.03277/) (2020)
- [Kornia: an Open Source Differentiable Computer Vision Library for PyTorch](https://mlsanity.com/paper/1910.02190/) (2019)
- [Contact and Human Dynamics from Monocular Video](https://mlsanity.com/paper/2007.11678/) (2020)
- [SNUG: Self-Supervised Neural Dynamic Garments](https://mlsanity.com/paper/2204.02219/) (2022)
- [Scalable Differentiable Physics for Learning and Control](https://mlsanity.com/paper/2007.02168/) (2020)
- [DiffCloth: Differentiable Cloth Simulation with Dry Frictional Contact](https://mlsanity.com/paper/2106.05306/) (2021)
- [Physics-as-Inverse-Graphics: Unsupervised Physical Parameter Estimation from Video](https://mlsanity.com/paper/1905.11169/) (2019)
- [Deep Physics-aware Inference of Cloth Deformation for Monocular Human Performance Capture](https://mlsanity.com/paper/2011.12866/) (2020)
- [IsMo-GAN: Adversarial Learning for Monocular Non-Rigid 3D Reconstruction](https://mlsanity.com/paper/1904.12144/) (2019)
- [HDM-Net: Monocular Non-Rigid 3D Reconstruction with Learned Deformation Model](https://mlsanity.com/paper/1803.10193/) (2018)
- [Geometry-Aware Network for Non-Rigid Shape Prediction from a Single View](https://mlsanity.com/paper/1809.10305/) (2018)
- [Sparse Surface Constraints for Combining Physics-based Elasticity Simulation and Correspondence-Free Object Reconstruction](https://mlsanity.com/paper/1910.01812/) (2019)
- [GAPS: Geometry-Aware, Physics-Based, Self-Supervised Neural Garment Draping](https://mlsanity.com/paper/2312.01490/) (2023)
- [Physics-guided Shape-from-Template: Monocular Video Perception through Neural Surrogate Models](https://mlsanity.com/paper/2311.12796/) (2023)
- [Better Together: Joint Reasoning for Non-rigid 3D Reconstruction with Specularities and Shading](https://mlsanity.com/paper/1708.01654/) (2017)
- [ROBUSfT: Robust Real-Time Shape-from-Template, a C++ Library](https://mlsanity.com/paper/2301.04037/) (2023)

## Abstract (arXiv)

Shape-from-Template (SfT) refers to the class of methods that reconstruct the 3D shape of a deforming object from images/videos using a 3D template. Traditional SfT methods require point correspondences between images and the texture of the 3D template in order to reconstruct 3D shapes from images/videos in real time. Their performance severely degrades when encountered with severe occlusions in the images because of the unavailability of correspondences. In contrast, modern SfT methods use a correspondence-free approach by incorporating deep neural networks to reconstruct 3D objects, thus requiring huge amounts of data for supervision. Recent advances use a fully unsupervised or self-supervised approach by combining differentiable physics and graphics to deform 3D template to match input images. In this paper, we propose an unsupervised SfT which uses only image observations: color features, gradients and silhouettes along with a mesh inextensibility constraint to reconstruct at a 400× faster pace than (best-performing) unsupervised SfT. Moreover, when it comes to generating finer details and severe occlusions, our method outperforms the existing methodologies by a large margin. Code is available at https://github.com/dvttran/nsft.