# Leverage Your Local and Global Representations: A New Self-Supervised Learning Strategy

arXiv: https://arxiv.org/abs/2203.17205 · HTML: https://mlsanity.com/paper/2203.17205/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 53 references that are in the map.
- First posted on arXiv 31 Mar 2022.

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Colorful Image Colorization](https://mlsanity.com/paper/1603.08511/) (2016)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [Unsupervised Deep Embedding for Clustering Analysis](https://mlsanity.com/paper/1511.06335/) (2015)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [Do Better ImageNet Models Transfer Better?](https://mlsanity.com/paper/1805.08974/) (2018)
- [Self-labelling via simultaneous clustering and representation learning](https://mlsanity.com/paper/1911.05371/) (2019)
- [Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem](https://mlsanity.com/paper/1812.05720/) (2018)
- [Joint Unsupervised Learning of Deep Representations and Image Clusters](https://mlsanity.com/paper/1604.03628/) (2016)
- [What makes instance discrimination good for transfer learning?](https://mlsanity.com/paper/2006.06606/) (2020)

## Built on by (most cited first, 30 of 30)

- [Know Your Self-supervised Learning: A Survey on Image-based Generative and Discriminative Training](https://mlsanity.com/paper/2305.13689/) (2023)
- [Anatomical Invariance Modeling and Semantic Alignment for Self-supervised Learning in 3D Medical Image Analysis](https://mlsanity.com/paper/2302.05615/) (2023)
- [Learning Natural Consistency Representation for Face Forgery Video Detection](https://mlsanity.com/paper/2407.10550/) (2024)
- [Spatiotemporal Self-supervised Learning for Point Clouds in the Wild](https://mlsanity.com/paper/2303.16235/) (2023)
- [Advancing Volumetric Medical Image Segmentation via Global-Local Masked Autoencoder](https://mlsanity.com/paper/2306.08913/) (2023)
- [MiM: Mask in Mask Self-Supervised Pre-Training for 3D Medical Image Analysis](https://mlsanity.com/paper/2404.15580/) (2024)
- [SINDER: Repairing the Singular Defects of DINOv2](https://mlsanity.com/paper/2407.16826/) (2024)
- [Representing Part-Whole Hierarchies in Foundation Models by Learning Localizability, Composability, and Decomposability from Anatomy via Self-Supervision](https://mlsanity.com/paper/2404.15672/) (2024)
- [Novel Class Discovery for Ultra-Fine-Grained Visual Categorization](https://mlsanity.com/paper/2405.06283/) (2024)
- [Refining CLIP's Spatial Awareness: A Visual-Centric Perspective](https://mlsanity.com/paper/2504.02328/) (2025)
- [Contrastive Learning with Consistent Representations](https://mlsanity.com/paper/2302.01541/) (2023)
- [S4: Self-Supervised Sensing Across the Spectrum](https://mlsanity.com/paper/2405.01656/) (2024)
- [Autoregressive Sequence Modeling for 3D Medical Image Representation](https://mlsanity.com/paper/2409.08691/) (2024)
- [De-coupling and De-positioning Dense Self-supervised Learning](https://mlsanity.com/paper/2303.16947/) (2023)
- [Joint Salient Object Detection and Camouflaged Object Detection via Uncertainty-aware Learning](https://mlsanity.com/paper/2307.04651/) (2023)
- [LeOCLR: Leveraging Original Images for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2403.06813/) (2024)
- [BiSSL: Enhancing the Alignment Between Self-Supervised Pretraining and Downstream Fine-Tuning via Bilevel Optimization](https://mlsanity.com/paper/2410.02387/) (2024)
- [Invariant Graph Learning Meets Information Bottleneck for Out-of-Distribution Generalization](https://mlsanity.com/paper/2408.01697/) (2024)
- [Patch-Wise Self-Supervised Visual Representation Learning: A Fine-Grained Approach](https://mlsanity.com/paper/2310.18651/) (2023)
- [Rethinking Samples Selection for Contrastive Learning: Mining of Potential Samples](https://mlsanity.com/paper/2311.00358/) (2023)
- [ACE: Anatomically Consistent Embeddings in Composition and Decomposition](https://mlsanity.com/paper/2501.10131/) (2025)
- [Implicit Contrastive Representation Learning with Guided Stop-gradient](https://mlsanity.com/paper/2503.09058/) (2025)
- [AlignDrive: Aligned Lateral-Longitudinal Planning for End-to-End Autonomous Driving](https://mlsanity.com/paper/2601.01762/) (2026)
- [Self-Supervised Learning from Non-Object Centric Images with a Geometric Transformation Sensitive Architecture](https://mlsanity.com/paper/2304.08014/) (2023)
- [Recent Advances of Local Mechanisms in Computer Vision: A Survey and Outlook of Recent Work](https://mlsanity.com/paper/2306.01929/) (2023)
- [OMH: Structured Sparsity via Optimally Matched Hierarchy for Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2403.06546/) (2024)
- [GenView++: Unifying Adaptive Generative Augmentation and Quality-Driven Supervision for Contrastive Representation Learning](https://mlsanity.com/paper/2509.23770/) (2025)
- [The Impact of Semantic Pairs on Self-Supervised Representation Learning](https://mlsanity.com/paper/2510.08722/) (2025)
- [Self-Supervised Learning with a Multi-Task Latent Space Objective](https://mlsanity.com/paper/2602.05845/) (2026)
- [Prior-Fitted and Frozen Scorers for Multimodal Contrastive Learning](https://mlsanity.com/paper/2610.14770/) (2026)

## Abstract (arXiv)

Self-supervised learning (SSL) methods aim to learn view-invariant representations by maximizing the similarity between the features extracted from different crops of the same image regardless of cropping size and content. In essence, this strategy ignores the fact that two crops may truly contain different image information, e.g., background and small objects, and thus tends to restrain the diversity of the learned representations. In this work, we address this issue by introducing a new self-supervised learning strategy, LoGo, that explicitly reasons about Local and Global crops. To achieve view invariance, LoGo encourages similarity between global crops from the same image, as well as between a global and a local crop. However, to correctly encode the fact that the content of smaller crops may differ entirely, LoGo promotes two local crops to have dissimilar representations, while being close to global crops. Our LoGo strategy can easily be applied to existing SSL methods. Our extensive experiments on a variety of datasets and using different self-supervised learning frameworks validate its superiority over existing approaches. Noticeably, we achieve better results than supervised models on transfer learning when using only 1/10 of the data.