# Single-view 3D Mesh Reconstruction for Seen and Unseen Categories

arXiv: https://arxiv.org/abs/2208.02676 · HTML: https://mlsanity.com/paper/2208.02676/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 46 references that are in the map.
- First posted on arXiv 4 Aug 2022.

## Builds on (most cited first, 30 of 38)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling](https://mlsanity.com/paper/1610.07584/) (2016)
- [AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generation](https://mlsanity.com/paper/1802.05384/) (2018)
- [PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization](https://mlsanity.com/paper/2004.00452/) (2020)
- [Neural 3D Mesh Renderer](https://mlsanity.com/paper/1711.07566/) (2017)
- [Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoning](https://mlsanity.com/paper/1904.01786/) (2019)
- [Learning Category-Specific Mesh Reconstruction from Image Collections](https://mlsanity.com/paper/1803.07549/) (2018)
- [What Do Single-view 3D Reconstruction Networks Learn?](https://mlsanity.com/paper/1905.03678/) (2019)
- [Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer](https://mlsanity.com/paper/1908.01210/) (2019)
- [Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling](https://mlsanity.com/paper/1804.04610/) (2018)
- [Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Image](https://mlsanity.com/paper/2002.12212/) (2020)
- [Self-supervised Single-view 3D Reconstruction via Semantic Consistency](https://mlsanity.com/paper/2003.06473/) (2020)
- [Deep Mesh Reconstruction from Single RGB Images via Topology Modification Networks](https://mlsanity.com/paper/1909.00321/) (2019)
- [SDF-SRN: Learning Signed Distance 3D Object Reconstruction from Static Images](https://mlsanity.com/paper/2010.10505/) (2020)
- [Shelf-Supervised Mesh Prediction in the Wild](https://mlsanity.com/paper/2102.06195/) (2021)
- [Learning to Reconstruct Shapes from Unseen Classes](https://mlsanity.com/paper/1812.11166/) (2018)
- [CoReNet: Coherent 3D scene reconstruction from a single RGB image](https://mlsanity.com/paper/2004.12989/) (2020)
- [3D Shape Reconstruction from 2D Images with Disentangled Attribute Flow](https://mlsanity.com/paper/2203.15190/) (2022)
- [DRWR: A Differentiable Renderer without Rendering for Unsupervised 3D Structure Learning from Silhouette Images](https://mlsanity.com/paper/2007.06127/) (2020)

## Built on by (most cited first, 10 of 10)

- [ReconX: Reconstruct Any Scene from Sparse Views with Video Diffusion Model](https://mlsanity.com/paper/2408.16767/) (2024)
- [ViewFusion: Towards Multi-View Consistency via Interpolated Denoising](https://mlsanity.com/paper/2402.18842/) (2024)
- [DreamCinema: Cinematic Transfer with Free Camera and 3D Character](https://mlsanity.com/paper/2408.12601/) (2024)
- [GeodesicPSIM: Predicting the Quality of Static Mesh with Texture Map via Geodesic Patch Similarity](https://mlsanity.com/paper/2308.04928/) (2023)
- [Neural Vector Fields: Generalizing Distance Vector Fields by Codebooks and Zero-Curl Regularization](https://mlsanity.com/paper/2309.01512/) (2023)
- [SeMv-3D: Towards Concurrency of Semantic and Multi-view Consistency in General Text-to-3D Generation](https://mlsanity.com/paper/2410.07658/) (2024)
- [CDI3D: Cross-guided Dense-view Interpolation for 3D Reconstruction](https://mlsanity.com/paper/2503.08005/) (2025)
- [HIVE: HIerarchical Volume Encoding for Neural Implicit Surface Reconstruction](https://mlsanity.com/paper/2408.01677/) (2024)
- [ROAD: Reciprocal-Objective Alignment of Discriminative Semantics for 3D Shape Generation](https://mlsanity.com/paper/2607.28581/) (2026)
- [Point Diffusion Mamba: Unified Diffusion-State-Space Modeling for Single-View 3D Reconstruction under Data Scarcity](https://mlsanity.com/paper/2609.25538/) (2026)

## Abstract (arXiv)

Single-view 3D object reconstruction is a fundamental and challenging computer vision task that aims at recovering 3D shapes from single-view RGB images. Most existing deep learning based reconstruction methods are trained and evaluated on the same categories, and they cannot work well when handling objects from novel categories that are not seen during training. Focusing on this issue, this paper tackles Single-view 3D Mesh Reconstruction, to study the model generalization on unseen categories and encourage models to reconstruct objects literally. Specifically, we propose an end-to-end two-stage network, GenMesh, to break the category boundaries in reconstruction. Firstly, we factorize the complicated image-to-mesh mapping into two simpler mappings, i.e., image-to-point mapping and point-to-mesh mapping, while the latter is mainly a geometric problem and less dependent on object categories. Secondly, we devise a local feature sampling strategy in 2D and 3D feature spaces to capture the local geometry shared across objects to enhance model generalization. Thirdly, apart from the traditional point-to-point supervision, we introduce a multi-view silhouette loss to supervise the surface generation process, which provides additional regularization and further relieves the overfitting problem. The experimental results show that our method significantly outperforms the existing works on the ShapeNet and Pix3D under different scenarios and various metrics, especially for novel objects. The project link is https://github.com/Wi-sc/GenMesh.