# Delving into Inter-Image Invariance for Unsupervised Visual Representations

arXiv: https://arxiv.org/abs/2008.11702 · HTML: https://mlsanity.com/paper/2008.11702/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Aug 2020.

## Built on by (most cited first, 30 of 37)

- [Exploring Cross-Image Pixel Contrast for Semantic Segmentation](https://mlsanity.com/paper/2101.11939/) (2021)
- [FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding](https://mlsanity.com/paper/2103.05950/) (2021)
- [Robust Audio-Visual Instance Discrimination](https://mlsanity.com/paper/2103.15916/) (2021)
- [Unsupervised Object-Level Representation Learning from Scene Images](https://mlsanity.com/paper/2106.11952/) (2021)
- [Learning with Mixture of Prototypes for Out-of-Distribution Detection](https://mlsanity.com/paper/2402.02653/) (2024)
- [Know Your Self-supervised Learning: A Survey on Image-based Generative and Discriminative Training](https://mlsanity.com/paper/2305.13689/) (2023)
- [Prompt Pre-Training with Twenty-Thousand Classes for Open-Vocabulary Visual Recognition](https://mlsanity.com/paper/2304.04704/) (2023)
- [MosaicFusion: Diffusion Models as Data Augmenters for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/2309.13042/) (2023)
- [Enhancing Self-supervised Video Representation Learning via Multi-level Feature Optimization](https://mlsanity.com/paper/2108.02183/) (2021)
- [Unsupervised Face Recognition using Unlabeled Synthetic Data](https://mlsanity.com/paper/2211.07371/) (2022)
- [Semi-supervised Contrastive Learning with Similarity Co-calibration](https://mlsanity.com/paper/2105.07387/) (2021)
- [Temperature as Uncertainty in Contrastive Learning](https://mlsanity.com/paper/2110.04403/) (2021)
- [Not All Semantics are Created Equal: Contrastive Self-supervised Learning with Automatic Temperature Individualization](https://mlsanity.com/paper/2305.11965/) (2023)
- [Clustering by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/2107.11635/) (2021)
- [Cluster-guided Asymmetric Contrastive Learning for Unsupervised Person Re-Identification](https://mlsanity.com/paper/2106.07846/) (2021)
- [CrIBo: Self-Supervised Learning via Cross-Image Object-Level Bootstrapping](https://mlsanity.com/paper/2310.07855/) (2023)
- [MarginNCE: Robust Sound Localization with a Negative Margin](https://mlsanity.com/paper/2211.01966/) (2022)
- [COARSE3D: Class-Prototypes for Contrastive Learning in Weakly-Supervised 3D Point Cloud Segmentation](https://mlsanity.com/paper/2210.01784/) (2022)
- [Significantly improving zero-shot X-ray pathology classification via fine-tuning pre-trained image-text encoders](https://mlsanity.com/paper/2212.07050/) (2022)
- [Does Negative Sampling Matter? A Review with Insights into its Theory and Applications](https://mlsanity.com/paper/2402.17238/) (2024)
- [Contextrast: Contextual Contrastive Learning for Semantic Segmentation](https://mlsanity.com/paper/2404.10633/) (2024)
- [InsCLR: Improving Instance Retrieval with Self-Supervision](https://mlsanity.com/paper/2112.01390/) (2021)
- [Correlational Image Modeling for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2303.12670/) (2023)
- [HVDistill: Transferring Knowledge from Images to Point Clouds via Unsupervised Hybrid-View Distillation](https://mlsanity.com/paper/2403.11817/) (2024)
- [Temporal Knowledge Consistency for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/2108.10668/) (2021)
- [End-to-end Alternating Optimization for Real-World Blind Super Resolution](https://mlsanity.com/paper/2308.08816/) (2023)
- [Test-Time Visual In-Context Tuning](https://mlsanity.com/paper/2503.21777/) (2025)
- [Self-Supervised Skeleton-Based Action Representation Learning: A Benchmark and Beyond](https://mlsanity.com/paper/2406.02978/) (2024)
- [Bridge the Gap between Supervised and Unsupervised Learning for Fine-Grained Classification](https://mlsanity.com/paper/2203.00441/) (2022)
- [Joint Prediction of Meningioma Grade and Brain Invasion via Task-Aware Contrastive Learning](https://mlsanity.com/paper/2209.01517/) (2022)

## Abstract (arXiv)

Contrastive learning has recently shown immense potential in unsupervised visual representation learning. Existing studies in this track mainly focus on intra-image invariance learning. The learning typically uses rich intra-image transformations to construct positive pairs and then maximizes agreement using a contrastive loss. The merits of inter-image invariance, conversely, remain much less explored. One major obstacle to exploit inter-image invariance is that it is unclear how to reliably construct inter-image positive pairs, and further derive effective supervision from them since no pair annotations are available. In this work, we present a comprehensive empirical study to better understand the role of inter-image invariance learning from three main constituting components: pseudo-label maintenance, sampling strategy, and decision boundary design. To facilitate the study, we introduce a unified and generic framework that supports the integration of unsupervised intra- and inter-image invariance learning. Through carefully-designed comparisons and analysis, multiple valuable observations are revealed: 1) online labels converge faster and perform better than offline labels; 2) semi-hard negative samples are more reliable and unbiased than hard negative samples; 3) a less stringent decision boundary is more favorable for inter-image invariance learning. With all the obtained recipes, our final model, namely InterCLR, shows consistent improvements over state-of-the-art intra-image invariance learning methods on multiple standard benchmarks. We hope this work will provide useful experience for devising effective unsupervised inter-image invariance learning. Code: https://github.com/open-mmlab/mmselfsup.