# Learning to Generate and Reconstruct 3D Meshes with only 2D Supervision

arXiv: https://arxiv.org/abs/1807.09259 · HTML: https://mlsanity.com/paper/1807.09259/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jul 2018.

## Built on by (most cited first, 12 of 12)

- [Improving Semantic Correspondence with Viewpoint-Guided Spherical Maps](https://mlsanity.com/paper/2312.13216/) (2023)
- [Shape, Pose, and Appearance from a Single Image via Bootstrapped Radiance Field Inversion](https://mlsanity.com/paper/2211.11674/) (2022)
- [SurfGen: Adversarial 3D Shape Synthesis with Explicit Surface Discriminators](https://mlsanity.com/paper/2201.00112/) (2022)
- [SSPU-Net: Self-Supervised Point Cloud Upsampling via Differentiable Rendering](https://mlsanity.com/paper/2108.00454/) (2021)
- [Structural Causal 3D Reconstruction](https://mlsanity.com/paper/2207.10156/) (2022)
- [Multi-View Reconstruction using Signed Ray Distance Functions (SRDF)](https://mlsanity.com/paper/2209.00082/) (2022)
- [Spectrum AUC Difference (SAUCD): Human-aligned 3D Shape Evaluation](https://mlsanity.com/paper/2403.01619/) (2024)
- [Multi-Category Mesh Reconstruction From Image Collections](https://mlsanity.com/paper/2110.11256/) (2021)
- [Style Agnostic 3D Reconstruction via Adversarial Style Transfer](https://mlsanity.com/paper/2110.10784/) (2021)
- [Two-stage Synthetic Supervising and Multi-view Consistency Self-supervising based Animal 3D Reconstruction by Single Image](https://mlsanity.com/paper/2311.13199/) (2023)
- [SRAM: Shape-Realism Alignment Metric for No Reference 3D Shape Evaluation](https://mlsanity.com/paper/2512.01373/) (2025)
- [Textured Geometry Evaluation: Perceptual 3D Textured Shape Metric via 3D Latent-Geometry Network](https://mlsanity.com/paper/2512.01380/) (2025)

## Abstract (arXiv)

We present a unified framework tackling two problems: class-specific 3D reconstruction from a single image, and generation of new 3D shape samples. These tasks have received considerable attention recently; however, existing approaches rely on 3D supervision, annotation of 2D images with keypoints or poses, and/or training with multiple views of each object instance. Our framework is very general: it can be trained in similar settings to these existing approaches, while also supporting weaker supervision scenarios. Importantly, it can be trained purely from 2D images, without ground-truth pose annotations, and with a single view per instance. We employ meshes as an output representation, instead of voxels used in most prior work. This allows us to exploit shading information during training, which previous 2D-supervised methods cannot. Thus, our method can learn to generate and reconstruct concave object classes. We evaluate our approach on synthetic data in various settings, showing that (i) it learns to disentangle shape from pose; (ii) using shading in the loss improves performance; (iii) our model is comparable or superior to state-of-the-art voxel-based approaches on quantitative metrics, while producing results that are visually more pleasing; (iv) it still performs well when given supervision weaker than in prior works.