# Learning Free-Form Deformations for 3D Object Reconstruction

arXiv: https://arxiv.org/abs/1803.10932 · HTML: https://mlsanity.com/paper/1803.10932/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Mar 2018.

## Built on by (most cited first, 20 of 20)

- [Joint Learning of 3D Shape Retrieval and Deformation](https://mlsanity.com/paper/2101.07889/) (2021)
- [DeepMetaHandles: Learning Deformation Meta-Handles of 3D Meshes with Biharmonic Coordinates](https://mlsanity.com/paper/2102.09105/) (2021)
- [360-GS: Layout-guided Panoramic Gaussian Splatting For Indoor Roaming](https://mlsanity.com/paper/2402.00763/) (2024)
- [U-RED: Unsupervised 3D Shape Retrieval and Deformation for Partial Point Clouds](https://mlsanity.com/paper/2308.06383/) (2023)
- [Towards Self-Supervised Category-Level Object Pose and Size Estimation](https://mlsanity.com/paper/2203.02884/) (2022)
- [Deep Generative Model for Efficient 3D Airfoil Parameterization and Generation](https://mlsanity.com/paper/2101.02744/) (2021)
- [ShapeMatcher: Self-Supervised Joint Shape Canonicalization, Segmentation, Retrieval and Deformation](https://mlsanity.com/paper/2311.11106/) (2023)
- [Learning Free-Form Deformation for 3D Face Reconstruction from In-The-Wild Images](https://mlsanity.com/paper/2105.14857/) (2021)
- [What Makes for Automatic Reconstruction of Pulmonary Segments](https://mlsanity.com/paper/2207.03078/) (2022)
- [KP-RED: Exploiting Semantic Keypoints for Joint 3D Shape Retrieval and Deformation](https://mlsanity.com/paper/2403.10099/) (2024)
- [IE-NeRF: Inpainting Enhanced Neural Radiance Fields in the Wild](https://mlsanity.com/paper/2407.10695/) (2024)
- [Is Single-View Mesh Reconstruction Ready for Robotics?](https://mlsanity.com/paper/2505.17966/) (2025)
- [Self-NeRF: A Self-Training Pipeline for Few-Shot Neural Radiance Fields](https://mlsanity.com/paper/2303.05775/) (2023)
- [PokeFlex: A Real-World Dataset of Volumetric Deformable Objects for Robotics](https://mlsanity.com/paper/2410.07688/) (2024)
- [Unsupervised Dense Deformation Embedding Network for Template-Free Shape Correspondence](https://mlsanity.com/paper/2108.11609/) (2021)
- [D-OccNet: Detailed 3D Reconstruction Using Cross-Domain Learning](https://mlsanity.com/paper/2104.13854/) (2021)
- [Pop-Out Motion: 3D-Aware Image Deformation via Learning the Shape Laplacian](https://mlsanity.com/paper/2203.15235/) (2022)
- [Learnable Triangulation for Deep Learning-based 3D Reconstruction of Objects of Arbitrary Topology from Single RGB Images](https://mlsanity.com/paper/2109.11844/) (2021)
- [3Deformer: A Common Framework for Image-Guided Mesh Deformation](https://mlsanity.com/paper/2307.09892/) (2023)
- [Geometry-Guided Modeling of Foundation Features Enables Generalizable Object Shape Deformation Learning](https://mlsanity.com/paper/2605.29661/) (2026)

## Abstract (arXiv)

Representing 3D shape in deep learning frameworks in an accurate, efficient and compact manner still remains an open challenge. Most existing work addresses this issue by employing voxel-based representations. While these approaches benefit greatly from advances in computer vision by generalizing 2D convolutions to the 3D setting, they also have several considerable drawbacks. The computational complexity of voxel-encodings grows cubically with the resolution thus limiting such representations to low-resolution 3D reconstruction. In an attempt to solve this problem, point cloud representations have been proposed. Although point clouds are more efficient than voxel representations as they only cover surfaces rather than volumes, they do not encode detailed geometric information about relationships between points. In this paper we propose a method to learn free-form deformations (FFD) for the task of 3D reconstruction from a single image. By learning to deform points sampled from a high-quality mesh, our trained model can be used to produce arbitrarily dense point clouds or meshes with fine-grained geometry. We evaluate our proposed framework on both synthetic and real-world data and achieve state-of-the-art results on point-cloud and volumetric metrics. Additionally, we qualitatively demonstrate its applicability to label transferring for 3D semantic segmentation.