# ROBUSfT: Robust Real-Time Shape-from-Template, a C++ Library

arXiv: https://arxiv.org/abs/2301.04037 · HTML: https://mlsanity.com/paper/2301.04037/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 50 references that are in the map.
- First posted on arXiv 10 Jan 2023.

## Builds on (most cited first, 11 of 11)

- [From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimation](https://mlsanity.com/paper/1907.10326/) (2019)
- [NeW CRFs: Neural Window Fully-connected CRFs for Monocular Depth Estimation](https://mlsanity.com/paper/2203.01502/) (2022)
- [High Quality Monocular Depth Estimation via Transfer Learning](https://mlsanity.com/paper/1812.11941/) (2018)
- [Attention Attention Everywhere: Monocular Depth Prediction with Skip Attention](https://mlsanity.com/paper/2210.09071/) (2022)
- [DefSLAM: Tracking and Mapping of Deforming Scenes from Monocular Sequences](https://mlsanity.com/paper/1908.08918/) (2019)
- [State of the Art in Dense Monocular Non-Rigid 3D Reconstruction](https://mlsanity.com/paper/2210.15664/) (2022)
- [φ-SfT: Shape-from-Template with a Physics-Based Deformation Model](https://mlsanity.com/paper/2203.11938/) (2022)
- [IsMo-GAN: Adversarial Learning for Monocular Non-Rigid 3D Reconstruction](https://mlsanity.com/paper/1904.12144/) (2019)
- [HDM-Net: Monocular Non-Rigid 3D Reconstruction with Learned Deformation Model](https://mlsanity.com/paper/1803.10193/) (2018)
- [Geometry-Aware Network for Non-Rigid Shape Prediction from a Single View](https://mlsanity.com/paper/1809.10305/) (2018)
- [Better Together: Joint Reasoning for Non-rigid 3D Reconstruction with Specularities and Shading](https://mlsanity.com/paper/1708.01654/) (2017)

## Built on by (most cited first, 6 of 6)

- [Shape-from-Template with Generalised Camera](https://mlsanity.com/paper/2508.13791/) (2025)
- [Textureless Deformable Surface Reconstruction with Invisible Markers](https://mlsanity.com/paper/2308.13678/) (2023)
- [Image-Guided Shape-from-Template Using Mesh Inextensibility Constraints](https://mlsanity.com/paper/2507.22699/) (2025)
- [SAFT: Shape and Appearance of Fabrics from Template via Differentiable Physical Simulations from Monocular Video](https://mlsanity.com/paper/2509.08828/) (2025)
- [Generalizing Shape-from-Template to Topological Changes](https://mlsanity.com/paper/2511.03459/) (2025)
- [Patch-based Representation and Learning for Efficient Deformation Modeling](https://mlsanity.com/paper/2601.05035/) (2026)

## Abstract (arXiv)

Tracking the 3D shape of a deforming object using only monocular 2D vision is a challenging problem. This is because one should (i) infer the 3D shape from a 2D image, which is a severely underconstrained problem, and (ii) implement the whole solution pipeline in real-time. The pipeline typically requires feature detection and matching, mismatch filtering, 3D shape inference and feature tracking algorithms. We propose ROBUSfT, a conventional pipeline based on a template containing the object's rest shape, texturemap and deformation law. ROBUSfT is ready-to-use, wide-baseline, capable of handling large deformations, fast up to 30 fps, free of training, and robust against partial occlusions and discontinuity in video frames. It outperforms the state-of-the-art methods in challenging datasets. ROBUSfT is implemented as a publicly available C++ library and we provide a tutorial on how to use it in https://github.com/mrshetab/ROBUSfT