Enhancing 2D Representation Learning with a 3D Prior
Not on the map. Why?
Abstract (arXiv)
Learning robust and effective representations of visual data is a fundamental task in computer vision. Traditionally, this is achieved by training models with labeled data which can be expensive to obtain. Self-supervised learning attempts to circumvent the requirement for labeled data by learning representations from raw unlabeled visual data alone. However, unlike humans who obtain rich 3D information from their binocular vision and through motion, the majority of current self-supervised methods are tasked with learning from monocular 2D image collections. This is noteworthy as it has been demonstrated that shape-centric visual processing is more robust compared to texture-biased automated methods. Inspired by this, we propose a new approach for strengthening existing self-supervised methods by explicitly enforcing a strong 3D structural prior directly into the model during training. Through experiments, across a range of datasets, we demonstrate that our 3D aware representations are more robust compared to conventional self-supervised baselines.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 59 of its 76 references that are in the map. First posted on arXiv 4 Jun 2024.
Builds on
59 of 76 references.
References · 59
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
- 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
- 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
- 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
38 more
- 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
- 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,842
- 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
- 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
- 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
- 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
- 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
- 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
- 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
- 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
- 2016Colorful Image Colorizationcited by 1,113
- 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
- 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
- 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
- 2017End-to-end Recovery of Human Shape and Posecited by 883
- 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
- 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
- 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
- 2023Scaling Vision Transformers to 22 Billion Parameterscited by 690
- 2019Self-supervised Visual Feature Learning with Deep Neural Networks: A Surveycited by 656
- 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
- 2018Taskonomy: Disentangling Task Transfer Learningcited by 631
- 2018DensePose: Dense Human Pose Estimation In The Wildcited by 573
- 2021Intriguing Properties of Vision Transformerscited by 446
- 2016Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescuecited by 368
- 2019Which Tasks Should Be Learned Together in Multi-task Learning?cited by 367
- 2014Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networkscited by 299
- 2019What Do Single-view 3D Reconstruction Networks Learn?cited by 259
- 2023MVImgNet: A Large-scale Dataset of Multi-view Imagescited by 219
- 2021Partial success in closing the gap between human and machine visioncited by 185
- 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
- 2021Benchmarking Representation Learning for Natural World Image Collectionscited by 167
- 2017Representation Learning by Learning to Countcited by 120
- 20223D Common Corruptions and Data Augmentationcited by 120
- 2022MagicPony: Learning Articulated 3D Animals in the Wildcited by 111
- 2021When Does Contrastive Visual Representation Learning Work?cited by 92
- 2023Battle of the Backbones: A Large-Scale Comparison of Pretrained Models across Computer Vision Taskscited by 72
- 2021Unsupervised Learning of 3D Object Categories from Videos in the Wildcited by 67
50 most cited of 59 All 59 (JSON)
Built on by
Citing papers per year: 2024: 1; 2025: 0; 2026: 2.
3 papers.