# Self-Supervised Viewpoint Learning From Image Collections

arXiv: https://arxiv.org/abs/2004.01793 · HTML: https://mlsanity.com/paper/2004.01793/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Apr 2020.

## Built on by (most cited first, 23 of 23)

- [GNeRF: GAN-based Neural Radiance Field without Posed Camera](https://mlsanity.com/paper/2103.15606/) (2021)
- [RayZer: A Self-supervised Large View Synthesis Model](https://mlsanity.com/paper/2505.00702/) (2025)
- [High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimization](https://mlsanity.com/paper/2211.15662/) (2022)
- [Neural Volumetric Memory for Visual Locomotion Control](https://mlsanity.com/paper/2304.01201/) (2023)
- [Video Autoencoder: self-supervised disentanglement of static 3D structure and motion](https://mlsanity.com/paper/2110.02951/) (2021)
- [PoseContrast: Class-Agnostic Object Viewpoint Estimation in the Wild with Pose-Aware Contrastive Learning](https://mlsanity.com/paper/2105.05643/) (2021)
- [Social Recommendation with Self-Supervised Metagraph Informax Network](https://mlsanity.com/paper/2110.03958/) (2021)
- [LARGE: Latent-Based Regression through GAN Semantics](https://mlsanity.com/paper/2107.11186/) (2021)
- [FisherMatch: Semi-Supervised Rotation Regression via Entropy-based Filtering](https://mlsanity.com/paper/2203.15765/) (2022)
- [ShapeClipper: Scalable 3D Shape Learning from Single-View Images via Geometric and CLIP-based Consistency](https://mlsanity.com/paper/2304.06247/) (2023)
- [Generative Modeling for Multi-task Visual Learning](https://mlsanity.com/paper/2106.13409/) (2021)
- [MonoNeRF: Learning Generalizable NeRFs from Monocular Videos without Camera Pose](https://mlsanity.com/paper/2210.07181/) (2022)
- [Planes vs. Chairs: Category-guided 3D shape learning without any 3D cues](https://mlsanity.com/paper/2204.10235/) (2022)
- [Contrastive Attention Network with Dense Field Estimation for Face Completion](https://mlsanity.com/paper/2112.10310/) (2021)
- [Semi-supervised Viewpoint Estimation with Geometry-aware Conditional Generation](https://mlsanity.com/paper/2104.01103/) (2021)
- [Self-Improving 4D Perception via Self-Distillation](https://mlsanity.com/paper/2604.08532/) (2026)
- [ViewNet: Unsupervised Viewpoint Estimation from Conditional Generation](https://mlsanity.com/paper/2212.00435/) (2022)
- [SSR: Semi-supervised Soft Rasterizer for single-view 2D to 3D Reconstruction](https://mlsanity.com/paper/2108.09593/) (2021)
- [Self-Supervised Object Detection via Generative Image Synthesis](https://mlsanity.com/paper/2110.09848/) (2021)
- [3D-Augmented Contrastive Knowledge Distillation for Image-based Object Pose Estimation](https://mlsanity.com/paper/2206.02531/) (2022)
- [ViewNeRF: Unsupervised Viewpoint Estimation Using Category-Level Neural Radiance Fields](https://mlsanity.com/paper/2212.00436/) (2022)
- [Accidental Turntables: Learning 3D Pose by Watching Objects Turn](https://mlsanity.com/paper/2212.06300/) (2022)
- [How Self-Supervised Learning Can be Used for Fine-Grained Head Pose Estimation?](https://mlsanity.com/paper/2108.04893/) (2021)

## Abstract (arXiv)

Training deep neural networks to estimate the viewpoint of objects requires large labeled training datasets. However, manually labeling viewpoints is notoriously hard, error-prone, and time-consuming. On the other hand, it is relatively easy to mine many unlabelled images of an object category from the internet, e.g., of cars or faces. We seek to answer the research question of whether such unlabeled collections of in-the-wild images can be successfully utilized to train viewpoint estimation networks for general object categories purely via self-supervision. Self-supervision here refers to the fact that the only true supervisory signal that the network has is the input image itself. We propose a novel learning framework which incorporates an analysis-by-synthesis paradigm to reconstruct images in a viewpoint aware manner with a generative network, along with symmetry and adversarial constraints to successfully supervise our viewpoint estimation network. We show that our approach performs competitively to fully-supervised approaches for several object categories like human faces, cars, buses, and trains. Our work opens up further research in self-supervised viewpoint learning and serves as a robust baseline for it. We open-source our code at https://github.com/NVlabs/SSV.