# Parametric Instance Classification for Unsupervised Visual Feature Learning

arXiv: https://arxiv.org/abs/2006.14618 · HTML: https://mlsanity.com/paper/2006.14618/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jun 2020.

## Built on by (most cited first, 30 of 41)

- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)
- [A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends](https://mlsanity.com/paper/2301.05712/) (2023)
- [On Feature Decorrelation in Self-Supervised Learning](https://mlsanity.com/paper/2105.00470/) (2021)
- [Self-Supervised Learning with Swin Transformers](https://mlsanity.com/paper/2105.04553/) (2021)
- [Revealing the Dark Secrets of Masked Image Modeling](https://mlsanity.com/paper/2205.13543/) (2022)
- [Large-scale Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2106.03149/) (2021)
- [When Does Contrastive Visual Representation Learning Work?](https://mlsanity.com/paper/2105.05837/) (2021)
- [Divide and Contrast: Self-supervised Learning from Uncurated Data](https://mlsanity.com/paper/2105.08054/) (2021)
- [Exploring the Equivalence of Siamese Self-Supervised Learning via A Unified Gradient Framework](https://mlsanity.com/paper/2112.05141/) (2021)
- [Improving Contrastive Learning by Visualizing Feature Transformation](https://mlsanity.com/paper/2108.02982/) (2021)
- [HCSC: Hierarchical Contrastive Selective Coding](https://mlsanity.com/paper/2202.00455/) (2022)
- [Know Your Self-supervised Learning: A Survey on Image-based Generative and Discriminative Training](https://mlsanity.com/paper/2305.13689/) (2023)
- [Enhancing Self-supervised Video Representation Learning via Multi-level Feature Optimization](https://mlsanity.com/paper/2108.02183/) (2021)
- [Training Vision Transformers with Only 2040 Images](https://mlsanity.com/paper/2201.10728/) (2022)
- [DropMAE: Learning Representations via Masked Autoencoders with Spatial-Attention Dropout for Temporal Matching Tasks](https://mlsanity.com/paper/2304.00571/) (2023)
- [Connecting Joint-Embedding Predictive Architecture with Contrastive Self-supervised Learning](https://mlsanity.com/paper/2410.19560/) (2024)
- [Finding Order in Chaos: A Novel Data Augmentation Method for Time Series in Contrastive Learning](https://mlsanity.com/paper/2309.13439/) (2023)
- [MimCo: Masked Image Modeling Pre-training with Contrastive Teacher](https://mlsanity.com/paper/2209.03063/) (2022)
- [What Variables Affect Out-of-Distribution Generalization in Pretrained Models?](https://mlsanity.com/paper/2405.15018/) (2024)
- [PTGB: Pre-Train Graph Neural Networks for Brain Network Analysis](https://mlsanity.com/paper/2305.14376/) (2023)
- [CaCo: Both Positive and Negative Samples are Directly Learnable via Cooperative-adversarial Contrastive Learning](https://mlsanity.com/paper/2203.14370/) (2022)
- [Rethinking Prototypical Contrastive Learning through Alignment, Uniformity and Correlation](https://mlsanity.com/paper/2210.10194/) (2022)
- [Understanding the Role of Equivariance in Self-supervised Learning](https://mlsanity.com/paper/2411.06508/) (2024)
- [Breaking Shortcut: Exploring Fully Convolutional Cycle-Consistency for Video Correspondence Learning](https://mlsanity.com/paper/2105.05838/) (2021)
- [Self-supervised Video Representation Learning with Cross-Stream Prototypical Contrasting](https://mlsanity.com/paper/2106.10137/) (2021)
- [Could Giant Pretrained Image Models Extract Universal Representations?](https://mlsanity.com/paper/2211.02043/) (2022)
- [Contrastive Learning with Consistent Representations](https://mlsanity.com/paper/2302.01541/) (2023)
- [Beyond Instance Discrimination: Relation-aware Contrastive Self-supervised Learning](https://mlsanity.com/paper/2211.01796/) (2022)
- [MeSa: Masked, Geometric, and Supervised Pre-training for Monocular Depth Estimation](https://mlsanity.com/paper/2310.04551/) (2023)
- [Label-Free Synthetic Pretraining of Object Detectors](https://mlsanity.com/paper/2208.04268/) (2022)

## Abstract (arXiv)

This paper presents parametric instance classification (PIC) for unsupervised visual feature learning. Unlike the state-of-the-art approaches which do instance discrimination in a dual-branch non-parametric fashion, PIC directly performs a one-branch parametric instance classification, revealing a simple framework similar to supervised classification and without the need to address the information leakage issue. We show that the simple PIC framework can be as effective as the state-of-the-art approaches, i.e. SimCLR and MoCo v2, by adapting several common component settings used in the state-of-the-art approaches. We also propose two novel techniques to further improve effectiveness and practicality of PIC: 1) a sliding-window data scheduler, instead of the previous epoch-based data scheduler, which addresses the extremely infrequent instance visiting issue in PIC and improves the effectiveness; 2) a negative sampling and weight update correction approach to reduce the training time and GPU memory consumption, which also enables application of PIC to almost unlimited training images. We hope that the PIC framework can serve as a simple baseline to facilitate future study.