# From Image Collections to Point Clouds with Self-supervised Shape and Pose Networks

arXiv: https://arxiv.org/abs/2005.01939 · HTML: https://mlsanity.com/paper/2005.01939/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 May 2020.

## Built on by (most cited first, 18 of 18)

- [PointGPT: Auto-regressively Generative Pre-training from Point Clouds](https://mlsanity.com/paper/2305.11487/) (2023)
- [MVD-Fusion: Single-view 3D via Depth-consistent Multi-view Generation](https://mlsanity.com/paper/2404.03656/) (2024)
- [Share With Thy Neighbors: Single-View Reconstruction by Cross-Instance Consistency](https://mlsanity.com/paper/2204.10310/) (2022)
- [ShapeClipper: Scalable 3D Shape Learning from Single-View Images via Geometric and CLIP-based Consistency](https://mlsanity.com/paper/2304.06247/) (2023)
- [Planes vs. Chairs: Category-guided 3D shape learning without any 3D cues](https://mlsanity.com/paper/2204.10235/) (2022)
- [Learning Canonical 3D Object Representation for Fine-Grained Recognition](https://mlsanity.com/paper/2108.04628/) (2021)
- [RGB2Point: 3D Point Cloud Generation from Single RGB Images](https://mlsanity.com/paper/2407.14979/) (2024)
- [Learning ABCs: Approximate Bijective Correspondence for isolating factors of variation with weak supervision](https://mlsanity.com/paper/2103.03240/) (2021)
- [Self-Supervised Object Detection via Generative Image Synthesis](https://mlsanity.com/paper/2110.09848/) (2021)
- [Parcel3D: Shape Reconstruction from Single RGB Images for Applications in Transportation Logistics](https://mlsanity.com/paper/2304.08994/) (2023)
- [Self-supervised Learning by View Synthesis](https://mlsanity.com/paper/2304.11330/) (2023)
- [Self-supervised Learning for Enhancing Geometrical Modeling in 3D-Aware Generative Adversarial Network](https://mlsanity.com/paper/2312.11856/) (2023)
- [Mitigating Prior Shape Bias in Point Clouds via Differentiable Center Learning](https://mlsanity.com/paper/2402.02088/) (2024)
- [Unsupervised Style-based Explicit 3D Face Reconstruction from Single Image](https://mlsanity.com/paper/2304.12455/) (2023)
- [Masked Generative Extractor for Synergistic Representation and 3D Generation of Point Clouds](https://mlsanity.com/paper/2406.17342/) (2024)
- [Glissando-Net: Deep sinGLe vIew category level poSe eStimation ANd 3D recOnstruction](https://mlsanity.com/paper/2501.14896/) (2025)
- [S2MNet: Speckle-To-Mesh Net for Three-Dimensional Cardiac Morphology Reconstruction via Echocardiogram](https://mlsanity.com/paper/2505.06105/) (2025)
- [Point-MF: Stabilizing One-Step Mean Flows for Single-Image Point Cloud Reconstruction](https://mlsanity.com/paper/2604.24586/) (2026)

## Abstract (arXiv)

Reconstructing 3D models from 2D images is one of the fundamental problems in computer vision. In this work, we propose a deep learning technique for 3D object reconstruction from a single image. Contrary to recent works that either use 3D supervision or multi-view supervision, we use only single view images with no pose information during training as well. This makes our approach more practical requiring only an image collection of an object category and the corresponding silhouettes. We learn both 3D point cloud reconstruction and pose estimation networks in a self-supervised manner, making use of differentiable point cloud renderer to train with 2D supervision. A key novelty of the proposed technique is to impose 3D geometric reasoning into predicted 3D point clouds by rotating them with randomly sampled poses and then enforcing cycle consistency on both 3D reconstructions and poses. In addition, using single-view supervision allows us to do test-time optimization on a given test image. Experiments on the synthetic ShapeNet and real-world Pix3D datasets demonstrate that our approach, despite using less supervision, can achieve competitive performance compared to pose-supervised and multi-view supervised approaches.