# Deep Mesh Reconstruction from Single RGB Images via Topology Modification Networks

arXiv: https://arxiv.org/abs/1909.00321 · HTML: https://mlsanity.com/paper/1909.00321/

## Key facts

- Cited by 106 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 1 Sep 2019.

## Built on by (most cited first, 30 of 106)

- [InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models](https://mlsanity.com/paper/2404.07191/) (2024)
- [Neural Kernel Surface Reconstruction](https://mlsanity.com/paper/2305.19590/) (2023)
- [Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networks](https://mlsanity.com/paper/2103.10429/) (2021)
- [Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks](https://mlsanity.com/paper/2108.07478/) (2021)
- [Holistic 3D Scene Understanding from a Single Image with Implicit Representation](https://mlsanity.com/paper/2103.06422/) (2021)
- [The Power of Points for Modeling Humans in Clothing](https://mlsanity.com/paper/2109.01137/) (2021)
- [SCALE: Modeling Clothed Humans with a Surface Codec of Articulated Local Elements](https://mlsanity.com/paper/2104.07660/) (2021)
- [DOVE: Learning Deformable 3D Objects by Watching Videos](https://mlsanity.com/paper/2107.10844/) (2021)
- [SA-ConvONet: Sign-Agnostic Optimization of Convolutional Occupancy Networks](https://mlsanity.com/paper/2105.03582/) (2021)
- [Neural Fields as Learnable Kernels for 3D Reconstruction](https://mlsanity.com/paper/2111.13674/) (2021)
- [DiffCAD: Weakly-Supervised Probabilistic CAD Model Retrieval and Alignment from an RGB Image](https://mlsanity.com/paper/2311.18610/) (2023)
- [3D Shape Reconstruction from 2D Images with Disentangled Attribute Flow](https://mlsanity.com/paper/2203.15190/) (2022)
- [Towards High-Fidelity Single-view Holistic Reconstruction of Indoor Scenes](https://mlsanity.com/paper/2207.08656/) (2022)
- [Surface Reconstruction from Point Clouds: A Survey and a Benchmark](https://mlsanity.com/paper/2205.02413/) (2022)
- [Deep Implicit Surface Point Prediction Networks](https://mlsanity.com/paper/2106.05779/) (2021)
- [Category-Level Metric Scale Object Shape and Pose Estimation](https://mlsanity.com/paper/2109.00326/) (2021)
- [Learned Vertex Descent: A New Direction for 3D Human Model Fitting](https://mlsanity.com/paper/2205.06254/) (2022)
- [Sketch and Text Guided Diffusion Model for Colored Point Cloud Generation](https://mlsanity.com/paper/2308.02874/) (2023)
- [Neural Template: Topology-aware Reconstruction and Disentangled Generation of 3D Meshes](https://mlsanity.com/paper/2206.04942/) (2022)
- [Learning Parallel Dense Correspondence from Spatio-Temporal Descriptors for Efficient and Robust 4D Reconstruction](https://mlsanity.com/paper/2103.16341/) (2021)
- [3D Pose Transfer with Correspondence Learning and Mesh Refinement](https://mlsanity.com/paper/2109.15025/) (2021)
- [3PSDF: Three-Pole Signed Distance Function for Learning Surfaces with Arbitrary Topologies](https://mlsanity.com/paper/2205.15572/) (2022)
- [Motion2VecSets: 4D Latent Vector Set Diffusion for Non-rigid Shape Reconstruction and Tracking](https://mlsanity.com/paper/2401.06614/) (2024)
- [Choose Your Simulator Wisely: A Review on Open-source Simulators for Autonomous Driving](https://mlsanity.com/paper/2311.11056/) (2023)
- [MOST-GAN: 3D Morphable StyleGAN for Disentangled Face Image Manipulation](https://mlsanity.com/paper/2111.01048/) (2021)
- [Ghost on the Shell: An Expressive Representation of General 3D Shapes](https://mlsanity.com/paper/2310.15168/) (2023)
- [CSDN: Cross-modal Shape-transfer Dual-refinement Network for Point Cloud Completion](https://mlsanity.com/paper/2208.00751/) (2022)
- [Dynamic Gaussians Mesh: Consistent Mesh Reconstruction from Dynamic Scenes](https://mlsanity.com/paper/2404.12379/) (2024)
- [Unsupervised 3D Pose Transfer with Cross Consistency and Dual Reconstruction](https://mlsanity.com/paper/2211.10278/) (2022)
- [ALTO: Alternating Latent Topologies for Implicit 3D Reconstruction](https://mlsanity.com/paper/2212.04096/) (2022)

## Abstract (arXiv)

Reconstructing the 3D mesh of a general object from a single image is now possible thanks to the latest advances of deep learning technologies. However, due to the nontrivial difficulty of generating a feasible mesh structure, the state-of-the-art approaches often simplify the problem by learning the displacements of a template mesh that deforms it to the target surface. Though reconstructing a 3D shape with complex topology can be achieved by deforming multiple mesh patches, it remains difficult to stitch the results to ensure a high meshing quality. In this paper, we present an end-to-end single-view mesh reconstruction framework that is able to generate high-quality meshes with complex topologies from a single genus-0 template mesh. The key to our approach is a novel progressive shaping framework that alternates between mesh deformation and topology modification. While a deformation network predicts the per-vertex translations that reduce the gap between the reconstructed mesh and the ground truth, a novel topology modification network is employed to prune the error-prone faces, enabling the evolution of topology. By iterating over the two procedures, one can progressively modify the mesh topology while achieving higher reconstruction accuracy. Moreover, a boundary refinement network is designed to refine the boundary conditions to further improve the visual quality of the reconstructed mesh. Extensive experiments demonstrate that our approach outperforms the current state-of-the-art methods both qualitatively and quantitatively, especially for the shapes with complex topologies.