ViewNet: Unsupervised Viewpoint Estimation from Conditional Generation
Not on the map. Why?
Abstract (arXiv)
Understanding the 3D world without supervision is currently a major challenge in computer vision as the annotations required to supervise deep networks for tasks in this domain are expensive to obtain on a large scale. In this paper, we address the problem of unsupervised viewpoint estimation. We formulate this as a self-supervised learning task, where image reconstruction provides the supervision needed to predict the camera viewpoint. Specifically, we make use of pairs of images of the same object at training time, from unknown viewpoints, to self-supervise training by combining the viewpoint information from one image with the appearance information from the other. We demonstrate that using a perspective spatial transformer allows efficient viewpoint learning, outperforming existing unsupervised approaches on synthetic data, and obtains competitive results on the challenging PASCAL3D+ dataset.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 52 of its 63 references that are in the map. First posted on arXiv 1 Dec 2022.
Builds on
52 of 63 references.
References · 52
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2017Mask R-CNNcited by 7,259
- 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
- 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
- 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
- 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
- 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
- 2015Spatial Transformer Networkscited by 1,423
- 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
- 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
38 more
- 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 460
- 2018Spherical CNNscited by 452
- 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 401
- 2017SSD-6D: Making RGB-based 3D detection and 6D pose estimation great againcited by 243
- 2017BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depthcited by 225
- 2017Real-Time Seamless Single Shot 6D Object Pose Predictioncited by 210
- 2019Texture Fields: Learning Texture Representations in Function Spacecited by 210
- 2016Unrolled Generative Adversarial Networkscited by 208
- 2017Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistencycited by 177
- 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
- 2019Implicit 3D Orientation Learning for 6D Object Detection from RGB Imagescited by 170
- 2020Single-Stage Semantic Segmentation from Image Labelscited by 167
- 2020Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotationscited by 155
- 2018Face Alignment in Full Pose Range: A 3D Total Solutioncited by 147
- 2015Deep Convolutional Inverse Graphics Networkcited by 121
- 20163D Shape Induction from 2D Views of Multiple Objectscited by 121
- 2018Unsupervised Learning of Object Landmarks through Conditional Image Generationcited by 113
- 2016Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervisioncited by 106
- 2016HyperFace: A Deep Multi-task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognitioncited by 103
- 2020Category Level Object Pose Estimation via Neural Analysis-by-Synthesiscited by 100
- 20176-DoF Object Pose from Semantic Keypointscited by 89
- 2018Unsupervised Learning of Shape and Pose with Differentiable Point Cloudscited by 89
- 2018Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoningcited by 79
- 2018Unsupervised Geometry-Aware Representation for 3D Human Pose Estimationcited by 73
- 2018Explorations in Homeomorphic Variational Auto-Encodingcited by 65
- 2014Viewpoints and Keypointscited by 59
- 2018Multi-view Consistency as Supervisory Signal for Learning Shape and Pose Predictioncited by 52
- 2018Deep Directional Statistics: Pose Estimation with Uncertainty Quantificationcited by 45
- 2018StarMap for Category-Agnostic Keypoint and Viewpoint Estimationcited by 42
- 2019Spherical Regression: Learning Viewpoints, Surface Normals and 3D Rotations on n-Spherescited by 37
- 2017Interpretable Transformations with Encoder-Decoder Networkscited by 33
- 20183D Pose Estimation and 3D Model Retrieval for Objects in the Wildcited by 31
- 2017Learning 3D Object Categories by Looking Around Themcited by 26
- 2019Transformable Bottleneck Networkscited by 23
- 2020Self-Supervised Viewpoint Learning From Image Collectionscited by 23
- 2017Recurrent 3D Pose Sequence Machinescited by 12
- 2018Cross-Domain 3D Equivariant Image Embeddingscited by 12
- 2020Cylindrical Convolutional Networks for Joint Object Detection and Viewpoint Estimationcited by 7
50 most cited of 52 All 52 (JSON)
Built on by
Citing papers per year: 2022: 1; 2023: 1; 2024: 1.
3 papers.