A Review of Deep Learning-Powered Mesh Reconstruction Methods
Not on the map. Why?
Abstract (arXiv)
With the recent advances in hardware and rendering techniques, 3D models have emerged everywhere in our life. Yet creating 3D shapes is arduous and requires significant professional knowledge. Meanwhile, Deep learning has enabled high-quality 3D shape reconstruction from various sources, making it a viable approach to acquiring 3D shapes with minimal effort. Importantly, to be used in common 3D applications, the reconstructed shapes need to be represented as polygonal meshes, which is a challenge for neural networks due to the irregularity of mesh tessellations. In this survey, we provide a comprehensive review of mesh reconstruction methods that are powered by machine learning. We first describe various representations for 3D shapes in the deep learning context. Then we review the development of 3D mesh reconstruction methods from voxels, point clouds, single images, and multi-view images. Finally, we identify several challenges in this field and propose potential future directions.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 158 of its 187 references that are in the map. First posted on arXiv 6 Mar 2023.
Builds on
158 of 187 references.
References · 158
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2022Generative Adversarial Networkscited by 5,603
- 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
- 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
- 2013Auto-Encoding Variational Bayescited by 3,980
- 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
- 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
- 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
- 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
38 more
- 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
- 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
- 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
- 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
- 2017Deep Image Priorcited by 1,097
- 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
- 2021Volume Rendering of Neural Implicit Surfacescited by 990
- 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
- 2020Convolutional Occupancy Networkscited by 760
- 2020Implicit Geometric Regularization for Learning Shapescited by 707
- 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
- 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
- 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
- 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 643
- 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
- 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
- 2021Neural Fields in Visual Computing and Beyondcited by 539
- 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
- 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 481
- 2022MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstructioncited by 467
- 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
- 2017Neural 3D Mesh Renderercited by 437
- 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
- 2019SAL: Sign Agnostic Learning of Shapes from Raw Datacited by 389
- 2020Local Implicit Grid Representations for 3D Scenescited by 372
- 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 363
- 2020Implicit Functions in Feature Space for 3D Shape Reconstruction and Completioncited by 343
- 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 332
- 2020Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstructioncited by 321
- 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
- 2021Neural RGB-D Surface Reconstructioncited by 312
- 2022MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architecturescited by 308
- 2019Convolutional Mesh Regression for Single-Image Human Shape Reconstructioncited by 270
- 2020PolyGen: An Autoregressive Generative Model of 3D Meshescited by 267
- 2021DeepCAD: A Deep Generative Network for Computer-Aided Design Modelscited by 266
- 2020Neural Unsigned Distance Fields for Implicit Function Learningcited by 265
- 2019What Do Single-view 3D Reconstruction Networks Learn?cited by 259
- 2019Multi-Garment Net: Learning to Dress 3D People from Imagescited by 244
50 most cited of 158 All 158 (JSON)
Built on by
Citing papers per year: 2023: 1; 2024: 0; 2025: 2.
3 papers.