# Better Together: Joint Reasoning for Non-rigid 3D Reconstruction with Specularities and Shading

arXiv: https://arxiv.org/abs/1708.01654 · HTML: https://mlsanity.com/paper/1708.01654/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Aug 2017.

## Built on by (most cited first, 6 of 6)

- [State of the Art in Dense Monocular Non-Rigid 3D Reconstruction](https://mlsanity.com/paper/2210.15664/) (2022)
- [ROBUSfT: Robust Real-Time Shape-from-Template, a C++ Library](https://mlsanity.com/paper/2301.04037/) (2023)
- [NRST: Non-rigid Surface Tracking from Monocular Video](https://mlsanity.com/paper/2107.02407/) (2021)
- [Textureless Deformable Surface Reconstruction with Invisible Markers](https://mlsanity.com/paper/2308.13678/) (2023)
- [Image-Guided Shape-from-Template Using Mesh Inextensibility Constraints](https://mlsanity.com/paper/2507.22699/) (2025)
- [SAFT: Shape and Appearance of Fabrics from Template via Differentiable Physical Simulations from Monocular Video](https://mlsanity.com/paper/2509.08828/) (2025)

## Abstract (arXiv)

We demonstrate the use of shape-from-shading (SfS) to improve both the quality and the robustness of 3D reconstruction of dynamic objects captured by a single camera. Unlike previous approaches that made use of SfS as a post-processing step, we offer a principled integrated approach that solves dynamic object tracking and reconstruction and SfS as a single unified cost function. Moving beyond Lambertian S f S , we propose a general approach that models both specularities and shading while simultaneously tracking and reconstructing general dynamic objects. Solving these problems jointly prevents the kinds of tracking failures which can not be recovered from by pipeline approaches. We show state-of-the-art results both qualitatively and quantitatively.