# IsMo-GAN: Adversarial Learning for Monocular Non-Rigid 3D Reconstruction

arXiv: https://arxiv.org/abs/1904.12144 · HTML: https://mlsanity.com/paper/1904.12144/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Apr 2019.

## Built on by (most cited first, 16 of 16)

- [State of the Art in Dense Monocular Non-Rigid 3D Reconstruction](https://mlsanity.com/paper/2210.15664/) (2022)
- [Recent Trends in 3D Reconstruction of General Non-Rigid Scenes](https://mlsanity.com/paper/2403.15064/) (2024)
- [Unbiased 4D: Monocular 4D Reconstruction with a Neural Deformation Model](https://mlsanity.com/paper/2206.08368/) (2022)
- [Differentiable Event Stream Simulator for Non-Rigid 3D Tracking](https://mlsanity.com/paper/2104.15139/) (2021)
- [φ-SfT: Shape-from-Template with a Physics-Based Deformation Model](https://mlsanity.com/paper/2203.11938/) (2022)
- [Decaf: Monocular Deformation Capture for Face and Hand Interactions](https://mlsanity.com/paper/2309.16670/) (2023)
- [HandVoxNet++: 3D Hand Shape and Pose Estimation using Voxel-Based Neural Networks](https://mlsanity.com/paper/2107.01205/) (2021)
- [Physics-guided Shape-from-Template: Monocular Video Perception through Neural Surrogate Models](https://mlsanity.com/paper/2311.12796/) (2023)
- [ROBUSfT: Robust Real-Time Shape-from-Template, a C++ Library](https://mlsanity.com/paper/2301.04037/) (2023)
- [Fast Non-Rigid Radiance Fields from Monocularized Data](https://mlsanity.com/paper/2212.01368/) (2022)
- [Thin-Shell-SfT: Fine-Grained Monocular Non-rigid 3D Surface Tracking with Neural Deformation Fields](https://mlsanity.com/paper/2503.19976/) (2025)
- [Textureless Deformable Surface Reconstruction with Invisible Markers](https://mlsanity.com/paper/2308.13678/) (2023)
- [Shape2.5D: A Dataset of Texture-less Surfaces for Depth and Normals Estimation](https://mlsanity.com/paper/2406.15831/) (2024)
- [Image-Guided Shape-from-Template Using Mesh Inextensibility Constraints](https://mlsanity.com/paper/2507.22699/) (2025)
- [SAFT: Shape and Appearance of Fabrics from Template via Differentiable Physical Simulations from Monocular Video](https://mlsanity.com/paper/2509.08828/) (2025)
- [Patch-based Representation and Learning for Efficient Deformation Modeling](https://mlsanity.com/paper/2601.05035/) (2026)

## Abstract (arXiv)

The majority of the existing methods for non-rigid 3D surface regression from monocular 2D images require an object template or point tracks over multiple frames as an input, and are still far from real-time processing rates. In this work, we present the Isometry-Aware Monocular Generative Adversarial Network (IsMo-GAN) - an approach for direct 3D reconstruction from a single image, trained for the deformation model in an adversarial manner on a light-weight synthetic dataset. IsMo-GAN reconstructs surfaces from real images under varying illumination, camera poses, textures and shading at over 250 Hz. In multiple experiments, it consistently outperforms several approaches in the reconstruction accuracy, runtime, generalisation to unknown surfaces and robustness to occlusions. In comparison to the state-of-the-art, we reduce the reconstruction error by 10-30% including the textureless case and our surfaces evince fewer artefacts qualitatively.