# Weakly supervised 3D Reconstruction with Adversarial Constraint

arXiv: https://arxiv.org/abs/1705.10904 · HTML: https://mlsanity.com/paper/1705.10904/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 May 2017.

## Built on by (most cited first, 13 of 13)

- [Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Data](https://mlsanity.com/paper/2306.07881/) (2023)
- [Learning Generative Models of Textured 3D Meshes from Real-World Images](https://mlsanity.com/paper/2103.15627/) (2021)
- [Pixel2Mesh++: 3D Mesh Generation and Refinement from Multi-View Images](https://mlsanity.com/paper/2204.09866/) (2022)
- [StrobeNet: Category-Level Multiview Reconstruction of Articulated Objects](https://mlsanity.com/paper/2105.08016/) (2021)
- [3D Building Reconstruction from Monocular Remote Sensing Images with Multi-level Supervisions](https://mlsanity.com/paper/2404.04823/) (2024)
- [Structural Causal 3D Reconstruction](https://mlsanity.com/paper/2207.10156/) (2022)
- [A Real World Dataset for Multi-view 3D Reconstruction](https://mlsanity.com/paper/2203.11397/) (2022)
- [Multi-Category Mesh Reconstruction From Image Collections](https://mlsanity.com/paper/2110.11256/) (2021)
- [SSR: Semi-supervised Soft Rasterizer for single-view 2D to 3D Reconstruction](https://mlsanity.com/paper/2108.09593/) (2021)
- [Self-Supervised Generative-Contrastive Learning of Multi-Modal Euclidean Input for 3D Shape Latent Representations: A Dynamic Switching Approach](https://mlsanity.com/paper/2301.04612/) (2023)
- [3D GANs and Latent Space: A comprehensive survey](https://mlsanity.com/paper/2304.03932/) (2023)
- [Two-stage Synthetic Supervising and Multi-view Consistency Self-supervising based Animal 3D Reconstruction by Single Image](https://mlsanity.com/paper/2311.13199/) (2023)
- [Glissando-Net: Deep sinGLe vIew category level poSe eStimation ANd 3D recOnstruction](https://mlsanity.com/paper/2501.14896/) (2025)

## Abstract (arXiv)

Supervised 3D reconstruction has witnessed a significant progress through the use of deep neural networks. However, this increase in performance requires large scale annotations of 2D/3D data. In this paper, we explore inexpensive 2D supervision as an alternative for expensive 3D CAD annotation. Specifically, we use foreground masks as weak supervision through a raytrace pooling layer that enables perspective projection and backpropagation. Additionally, since the 3D reconstruction from masks is an ill posed problem, we propose to constrain the 3D reconstruction to the manifold of unlabeled realistic 3D shapes that match mask observations. We demonstrate that learning a log-barrier solution to this constrained optimization problem resembles the GAN objective, enabling the use of existing tools for training GANs. We evaluate and analyze the manifold constrained reconstruction on various datasets for single and multi-view reconstruction of both synthetic and real images.