# CAPNet: Continuous Approximation Projection For 3D Point Cloud Reconstruction Using 2D Supervision

arXiv: https://arxiv.org/abs/1811.11731 · HTML: https://mlsanity.com/paper/1811.11731/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Nov 2018.

## Built on by (most cited first, 13 of 13)

- [3D Shape Reconstruction from 2D Images with Disentangled Attribute Flow](https://mlsanity.com/paper/2203.15190/) (2022)
- [Towards High-Fidelity Single-view Holistic Reconstruction of Indoor Scenes](https://mlsanity.com/paper/2207.08656/) (2022)
- [Unsupervised Learning of Fine Structure Generation for 3D Point Clouds by 2D Projection Matching](https://mlsanity.com/paper/2108.03746/) (2021)
- [Discovering 3D Parts from Image Collections](https://mlsanity.com/paper/2107.13629/) (2021)
- [Multi-View Reconstruction using Signed Ray Distance Functions (SRDF)](https://mlsanity.com/paper/2209.00082/) (2022)
- [Leveraging Single-View Images for Unsupervised 3D Point Cloud Completion](https://mlsanity.com/paper/2212.00564/) (2022)
- [Flow-based GAN for 3D Point Cloud Generation from a Single Image](https://mlsanity.com/paper/2210.04072/) (2022)
- [Learning Local Pattern Modularization for Point Cloud Reconstruction from Unseen Classes](https://mlsanity.com/paper/2408.14279/) (2024)
- [Visual Enhanced 3D Point Cloud Reconstruction from A Single Image](https://mlsanity.com/paper/2108.07685/) (2021)
- [Prototype-Aware Heterogeneous Task for Point Cloud Completion](https://mlsanity.com/paper/2209.01733/) (2022)
- [Fast-Image2Point: Towards Real-Time Point Cloud Reconstruction of a Single Image using 3D Supervision](https://mlsanity.com/paper/2209.10029/) (2022)
- [UNeR3D: Versatile and Scalable 3D RGB Point Cloud Generation from 2D Images in Unsupervised Reconstruction](https://mlsanity.com/paper/2312.06706/) (2023)
- [A Modular Framework for Single-View 3D Reconstruction of Indoor Environments](https://mlsanity.com/paper/2512.17955/) (2025)

## Abstract (arXiv)

Knowledge of 3D properties of objects is a necessity in order to build effective computer vision systems. However, lack of large scale 3D datasets can be a major constraint for data-driven approaches in learning such properties. We consider the task of single image 3D point cloud reconstruction, and aim to utilize multiple foreground masks as our supervisory data to alleviate the need for large scale 3D datasets. A novel differentiable projection module, called 'CAPNet', is introduced to obtain such 2D masks from a predicted 3D point cloud. The key idea is to model the projections as a continuous approximation of the points in the point cloud. To overcome the challenges of sparse projection maps, we propose a loss formulation termed 'affinity loss' to generate outlier-free reconstructions. We significantly outperform the existing projection based approaches on a large-scale synthetic dataset. We show the utility and generalizability of such a 2D supervised approach through experiments on a real-world dataset, where lack of 3D data can be a serious concern. To further enhance the reconstructions, we also propose a test stage optimization procedure to obtain reconstructions that display high correspondence with the observed input image.