# Unsupervised Representation Learning by InvariancePropagation

arXiv: https://arxiv.org/abs/2010.11694 · HTML: https://mlsanity.com/paper/2010.11694/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Oct 2020.

## Built on by (most cited first, 18 of 18)

- [On Feature Decorrelation in Self-Supervised Learning](https://mlsanity.com/paper/2105.00470/) (2021)
- [Can contrastive learning avoid shortcut solutions?](https://mlsanity.com/paper/2106.11230/) (2021)
- [Large-scale Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2106.03149/) (2021)
- [Mean Shift for Self-Supervised Learning](https://mlsanity.com/paper/2105.07269/) (2021)
- [Improving Contrastive Learning by Visualizing Feature Transformation](https://mlsanity.com/paper/2108.02982/) (2021)
- [Gaussian Mixture Variational Autoencoder with Contrastive Learning for Multi-Label Classification](https://mlsanity.com/paper/2112.00976/) (2021)
- [Contrastive Attraction and Contrastive Repulsion for Representation Learning](https://mlsanity.com/paper/2105.03746/) (2021)
- [GeomCA: Geometric Evaluation of Data Representations](https://mlsanity.com/paper/2105.12486/) (2021)
- [Constrained Mean Shift Using Distant Yet Related Neighbors for Representation Learning](https://mlsanity.com/paper/2112.04607/) (2021)
- [Unsupervised Visual Representation Learning via Mutual Information Regularized Assignment](https://mlsanity.com/paper/2211.02284/) (2022)
- [Improving Self-supervised Learning with Automated Unsupervised Outlier Arbitration](https://mlsanity.com/paper/2112.08132/) (2021)
- [Beyond Instance Discrimination: Relation-aware Contrastive Self-supervised Learning](https://mlsanity.com/paper/2211.01796/) (2022)
- [Self-supervised Video Representation Learning with Cascade Positive Retrieval](https://mlsanity.com/paper/2201.07989/) (2022)
- [CLAWSAT: Towards Both Robust and Accurate Code Models](https://mlsanity.com/paper/2211.11711/) (2022)
- [Unsupervised Cognition](https://mlsanity.com/paper/2409.18624/) (2024)
- [The Influences of Color and Shape Features in Visual Contrastive Learning](https://mlsanity.com/paper/2301.12459/) (2023)
- [Multi-view Feature Extraction based on Dual Contrastive Head](https://mlsanity.com/paper/2302.03932/) (2023)
- [Multi-view Feature Extraction based on Triple Contrastive Heads](https://mlsanity.com/paper/2303.12615/) (2023)

## Abstract (arXiv)

Unsupervised learning methods based on contrastive learning have drawn increasing attention and achieved promising results. Most of them aim to learn representations invariant to instance-level variations, which are provided by different views of the same instance. In this paper, we propose Invariance Propagation to focus on learning representations invariant to category-level variations, which are provided by different instances from the same category. Our method recursively discovers semantically consistent samples residing in the same high-density regions in representation space. We demonstrate a hard sampling strategy to concentrate on maximizing the agreement between the anchor sample and its hard positive samples, which provide more intra-class variations to help capture more abstract invariance. As a result, with a ResNet-50 as the backbone, our method achieves 71.3% top-1 accuracy on ImageNet linear classification and 78.2% top-5 accuracy fine-tuning on only 1% labels, surpassing previous results. We also achieve state-of-the-art performance on other downstream tasks, including linear classification on Places205 and Pascal VOC, and transfer learning on small scale datasets.