SSR: Semi-supervised Soft Rasterizer for single-view 2D to 3D Reconstruction
Not on the map. Why?
Abstract (arXiv)
Recent work has made significant progress in learning object meshes with weak supervision. Soft Rasterization methods have achieved accurate 3D reconstruction from 2D images with viewpoint supervision only. In this work, we further reduce the labeling effort by allowing such 3D reconstruction methods leverage unlabeled images. In order to obtain the viewpoints for these unlabeled images, we propose to use a Siamese network that takes two images as input and outputs whether they correspond to the same viewpoint. During training, we minimize the cross entropy loss to maximize the probability of predicting whether a pair of images belong to the same viewpoint or not. To get the viewpoint of a new image, we compare it against different viewpoints obtained from the training samples and select the viewpoint with the highest matching probability. We finally label the unlabeled images with the most confident predicted viewpoint and train a deep network that has a differentiable rasterization layer. Our experiments show that even labeling only two objects yields significant improvement in IoU for ShapeNet when leveraging unlabeled examples. Code is available at https://github.com/IssamLaradji/SSR.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 28 of its 41 references that are in the map. First posted on arXiv 21 Aug 2021.
Builds on
28 of 41 references.
References · 28
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
- 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
- 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
- 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
- 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
- 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
- 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
- 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 721
- 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
- 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
- 2019Deep Learning for 3D Point Clouds: A Surveycited by 557
16 more
- 2017Neural 3D Mesh Renderercited by 437
- 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
- 2019Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderercited by 221
- 2021Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Datacited by 214
- 2016Learning a Predictable and Generative Vector Representation for Objectscited by 173
- 2020Self-supervised Single-view 3D Reconstruction via Semantic Consistencycited by 127
- 2016Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervisioncited by 106
- 2020Embedding Propagation: Smoother Manifold for Few-Shot Classificationcited by 79
- 2016Unsupervised Learning of 3D Structure from Imagescited by 67
- 2018Multi-view Consistency as Supervisory Signal for Learning Shape and Pose Predictioncited by 52
- 20173D Sketching using Multi-View Deep Volumetric Predictioncited by 36
- 2020Self-Supervised Viewpoint Learning From Image Collectionscited by 23
- 2017Weakly supervised 3D Reconstruction with Adversarial Constraintcited by 13
- 2018A Semi-Supervised Data Augmentation Approach using 3D Graphical Enginescited by 5
- 2020Pix2Shape: Towards Unsupervised Learning of 3D Scenes from Images using a View-based Representationcited by 4
- 20183D Shape Reconstruction from a Single 2D Image via 2D-3D Self-Consistencycited by 1
Built on by
Citing papers per year: 2022: 1; 2023: 1.
2 papers.