# Synthesizing 3D Shapes from Silhouette Image Collections using Multi-projection Generative Adversarial Networks

arXiv: https://arxiv.org/abs/1906.03841 · HTML: https://mlsanity.com/paper/1906.03841/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2019.

## Built on by (most cited first, 14 of 14)

- [EpiGRAF: Rethinking training of 3D GANs](https://mlsanity.com/paper/2206.10535/) (2022)
- [SDF-StyleGAN: Implicit SDF-Based StyleGAN for 3D Shape Generation](https://mlsanity.com/paper/2206.12055/) (2022)
- [DeepMetaHandles: Learning Deformation Meta-Handles of 3D Meshes with Biharmonic Coordinates](https://mlsanity.com/paper/2102.09105/) (2021)
- [Generative Modelling of BRDF Textures from Flash Images](https://mlsanity.com/paper/2102.11861/) (2021)
- [Indoor Scene Generation from a Collection of Semantic-Segmented Depth Images](https://mlsanity.com/paper/2108.09022/) (2021)
- [Controllable Text-to-3D Generation via Surface-Aligned Gaussian Splatting](https://mlsanity.com/paper/2403.09981/) (2024)
- [Discovering 3D Parts from Image Collections](https://mlsanity.com/paper/2107.13629/) (2021)
- [FakeNews: GAN-based generation of realistic 3D volumetric data -- A systematic review and taxonomy](https://mlsanity.com/paper/2207.01390/) (2022)
- [MVControl: Adding Conditional Control to Multi-view Diffusion for Controllable Text-to-3D Generation](https://mlsanity.com/paper/2311.14494/) (2023)
- [ET3D: Efficient Text-to-3D Generation via Multi-View Distillation](https://mlsanity.com/paper/2311.15561/) (2023)
- [Understanding Pixel-level 2D Image Semantics with 3D Keypoint Knowledge Engine](https://mlsanity.com/paper/2111.10817/) (2021)
- [DreamPBR: Text-driven Generation of High-resolution SVBRDF with Multi-modal Guidance](https://mlsanity.com/paper/2404.14676/) (2024)
- [3D GANs and Latent Space: A comprehensive survey](https://mlsanity.com/paper/2304.03932/) (2023)
- [LPA3D: 3D Room-Level Scene Generation from In-the-Wild Images](https://mlsanity.com/paper/2504.02337/) (2025)

## Abstract (arXiv)

We present a new weakly supervised learning-based method for generating novel category-specific 3D shapes from unoccluded image collections. Our method is weakly supervised and only requires silhouette annotations from unoccluded, category-specific objects. Our method does not require access to the object's 3D shape, multiple observations per object from different views, intra-image pixel-correspondences, or any view annotations. Key to our method is a novel multi-projection generative adversarial network (MP-GAN) that trains a 3D shape generator to be consistent with multiple 2D projections of the 3D shapes, and without direct access to these 3D shapes. This is achieved through multiple discriminators that encode the distribution of 2D projections of the 3D shapes seen from a different views. Additionally, to determine the view information for each silhouette image, we also train a view prediction network on visualizations of 3D shapes synthesized by the generator. We iteratively alternate between training the generator and training the view prediction network. We validate our multi-projection GAN on both synthetic and real image datasets. Furthermore, we also show that multi-projection GANs can aid in learning other high-dimensional distributions from lower dimensional training datasets, such as material-class specific spatially varying reflectance properties from images.