# DeformNet: Free-Form Deformation Network for 3D Shape Reconstruction from a Single Image

arXiv: https://arxiv.org/abs/1708.04672 · HTML: https://mlsanity.com/paper/1708.04672/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Aug 2017.

## Built on by (most cited first, 25 of 25)

- [3D Shape Generation and Completion through Point-Voxel Diffusion](https://mlsanity.com/paper/2104.03670/) (2021)
- [DiT-3D: Exploring Plain Diffusion Transformers for 3D Shape Generation](https://mlsanity.com/paper/2307.01831/) (2023)
- [3D Semantic Scene Completion: a Survey](https://mlsanity.com/paper/2103.07466/) (2021)
- [Towards High-Fidelity Single-view Holistic Reconstruction of Indoor Scenes](https://mlsanity.com/paper/2207.08656/) (2022)
- [Joint Learning of 3D Shape Retrieval and Deformation](https://mlsanity.com/paper/2101.07889/) (2021)
- [Category-Level Metric Scale Object Shape and Pose Estimation](https://mlsanity.com/paper/2109.00326/) (2021)
- [DeepMetaHandles: Learning Deformation Meta-Handles of 3D Meshes with Biharmonic Coordinates](https://mlsanity.com/paper/2102.09105/) (2021)
- [Learning Whole Heart Mesh Generation From Patient Images For Computational Simulations](https://mlsanity.com/paper/2203.10517/) (2022)
- [Pixel2Mesh++: 3D Mesh Generation and Refinement from Multi-View Images](https://mlsanity.com/paper/2204.09866/) (2022)
- [U-RED: Unsupervised 3D Shape Retrieval and Deformation for Partial Point Clouds](https://mlsanity.com/paper/2308.06383/) (2023)
- [Towards Self-Supervised Category-Level Object Pose and Size Estimation](https://mlsanity.com/paper/2203.02884/) (2022)
- [Deep Generative Model for Efficient 3D Airfoil Parameterization and Generation](https://mlsanity.com/paper/2101.02744/) (2021)
- [FrePolad: Frequency-Rectified Point Latent Diffusion for Point Cloud Generation](https://mlsanity.com/paper/2311.12090/) (2023)
- [ShapeMatcher: Self-Supervised Joint Shape Canonicalization, Segmentation, Retrieval and Deformation](https://mlsanity.com/paper/2311.11106/) (2023)
- [Learning Free-Form Deformation for 3D Face Reconstruction from In-The-Wild Images](https://mlsanity.com/paper/2105.14857/) (2021)
- [KP-RED: Exploiting Semantic Keypoints for Joint 3D Shape Retrieval and Deformation](https://mlsanity.com/paper/2403.10099/) (2024)
- [LASA: Instance Reconstruction from Real Scans using A Large-scale Aligned Shape Annotation Dataset](https://mlsanity.com/paper/2312.12418/) (2023)
- [Style Agnostic 3D Reconstruction via Adversarial Style Transfer](https://mlsanity.com/paper/2110.10784/) (2021)
- [Visual Enhanced 3D Point Cloud Reconstruction from A Single Image](https://mlsanity.com/paper/2108.07685/) (2021)
- [Learnable Triangulation for Deep Learning-based 3D Reconstruction of Objects of Arbitrary Topology from Single RGB Images](https://mlsanity.com/paper/2109.11844/) (2021)
- [Fast-Image2Point: Towards Real-Time Point Cloud Reconstruction of a Single Image using 3D Supervision](https://mlsanity.com/paper/2209.10029/) (2022)
- [Deformable Voxel Grids for Shape Comparisons](https://mlsanity.com/paper/2211.11609/) (2022)
- [3Deformer: A Common Framework for Image-Guided Mesh Deformation](https://mlsanity.com/paper/2307.09892/) (2023)
- [Guided and Unguided Conditional Diffusion Mechanisms for Structured and Semantically-Aware 3D Point Cloud Generation](https://mlsanity.com/paper/2509.17206/) (2025)
- [Geometry-Guided Modeling of Foundation Features Enables Generalizable Object Shape Deformation Learning](https://mlsanity.com/paper/2605.29661/) (2026)

## Abstract (arXiv)

3D reconstruction from a single image is a key problem in multiple applications ranging from robotic manipulation to augmented reality. Prior methods have tackled this problem through generative models which predict 3D reconstructions as voxels or point clouds. However, these methods can be computationally expensive and miss fine details. We introduce a new differentiable layer for 3D data deformation and use it in DeformNet to learn a model for 3D reconstruction-through-deformation. DeformNet takes an image input, searches the nearest shape template from a database, and deforms the template to match the query image. We evaluate our approach on the ShapeNet dataset and show that - (a) the Free-Form Deformation layer is a powerful new building block for Deep Learning models that manipulate 3D data (b) DeformNet uses this FFD layer combined with shape retrieval for smooth and detail-preserving 3D reconstruction of qualitatively plausible point clouds with respect to a single query image (c) compared to other state-of-the-art 3D reconstruction methods, DeformNet quantitatively matches or outperforms their benchmarks by significant margins. For more information, visit: https://deformnet-site.github.io/DeformNet-website/ .