# Spatially Consistent Representation Learning

arXiv: https://arxiv.org/abs/2103.06122 · HTML: https://mlsanity.com/paper/2103.06122/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 46 references that are in the map.
- First posted on arXiv 10 Mar 2021.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [Self-Supervised Learning of Pretext-Invariant Representations](https://mlsanity.com/paper/1912.01991/) (2019)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [Data-Efficient Image Recognition with Contrastive Predictive Coding](https://mlsanity.com/paper/1905.09272/) (2019)
- [Sparse R-CNN: End-to-End Object Detection with Learnable Proposals](https://mlsanity.com/paper/2011.12450/) (2020)
- [Rethinking ImageNet Pre-training](https://mlsanity.com/paper/1811.08883/) (2018)
- [Learning Representations for Automatic Colorization](https://mlsanity.com/paper/1603.06668/) (2016)
- [Large Scale Adversarial Representation Learning](https://mlsanity.com/paper/1907.02544/) (2019)
- [Multi-task Self-Supervised Visual Learning](https://mlsanity.com/paper/1708.07860/) (2017)
- [LF-Net: Learning Local Features from Images](https://mlsanity.com/paper/1805.09662/) (2018)
- [Unsupervised Learning of Dense Visual Representations](https://mlsanity.com/paper/2011.05499/) (2020)

## Built on by (most cited first, 30 of 64)

- [Benchmarking Self-Supervised Learning on Diverse Pathology Datasets](https://mlsanity.com/paper/2212.04690/) (2022)
- [Large-scale Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2106.03149/) (2021)
- [Contrastive Learning with Stronger Augmentations](https://mlsanity.com/paper/2104.07713/) (2021)
- [Sparse DETR: Efficient End-to-End Object Detection with Learnable Sparsity](https://mlsanity.com/paper/2111.14330/) (2021)
- [Self-Supervised Learning of Object Parts for Semantic Segmentation](https://mlsanity.com/paper/2204.13101/) (2022)
- [DETReg: Unsupervised Pretraining with Region Priors for Object Detection](https://mlsanity.com/paper/2106.04550/) (2021)
- [Unsupervised Object-Level Representation Learning from Scene Images](https://mlsanity.com/paper/2106.11952/) (2021)
- [Masked Image Modeling with Denoising Contrast](https://mlsanity.com/paper/2205.09616/) (2022)
- [Point-Level Region Contrast for Object Detection Pre-Training](https://mlsanity.com/paper/2202.04639/) (2022)
- [UniVIP: A Unified Framework for Self-Supervised Visual Pre-training](https://mlsanity.com/paper/2203.06965/) (2022)
- [Learning Where to Learn in Cross-View Self-Supervised Learning](https://mlsanity.com/paper/2203.14898/) (2022)
- [Location-Aware Self-Supervised Transformers for Semantic Segmentation](https://mlsanity.com/paper/2212.02400/) (2022)
- [Boundary-aware Self-supervised Learning for Video Scene Segmentation](https://mlsanity.com/paper/2201.05277/) (2022)
- [RecursiveMix: Mixed Learning with History](https://mlsanity.com/paper/2203.06844/) (2022)
- [AlignDet: Aligning Pre-training and Fine-tuning in Object Detection](https://mlsanity.com/paper/2307.11077/) (2023)
- [R-MAE: Regions Meet Masked Autoencoders](https://mlsanity.com/paper/2306.05411/) (2023)
- [SERE: Exploring Feature Self-relation for Self-supervised Transformer](https://mlsanity.com/paper/2206.05184/) (2022)
- [DTG-SSOD: Dense Teacher Guidance for Semi-Supervised Object Detection](https://mlsanity.com/paper/2207.05536/) (2022)
- [Self-supervised Facial Action Unit Detection with Region and Relation Learning](https://mlsanity.com/paper/2303.05708/) (2023)
- [RegionCL: Can Simple Region Swapping Contribute to Contrastive Learning?](https://mlsanity.com/paper/2111.12309/) (2021)
- [ConCL: Concept Contrastive Learning for Dense Prediction Pre-training in Pathology Images](https://mlsanity.com/paper/2207.06733/) (2022)
- [Correlational Image Modeling for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2303.12670/) (2023)
- [Demystifying Unsupervised Semantic Correspondence Estimation](https://mlsanity.com/paper/2207.05054/) (2022)
- [Siamese Prototypical Contrastive Learning](https://mlsanity.com/paper/2208.08819/) (2022)
- [FLSL: Feature-level Self-supervised Learning](https://mlsanity.com/paper/2306.06203/) (2023)
- [Dense Siamese Network for Dense Unsupervised Learning](https://mlsanity.com/paper/2203.11075/) (2022)
- [Self-supervised Learning with Local Contrastive Loss for Detection and Semantic Segmentation](https://mlsanity.com/paper/2207.04398/) (2022)
- [Semantics-Consistent Feature Search for Self-Supervised Visual Representation Learning](https://mlsanity.com/paper/2212.06486/) (2022)
- [SCE-MAE: Selective Correspondence Enhancement with Masked Autoencoder for Self-Supervised Landmark Estimation](https://mlsanity.com/paper/2405.18322/) (2024)
- [Sound and Visual Representation Learning with Multiple Pretraining Tasks](https://mlsanity.com/paper/2201.01046/) (2022)

## Abstract (arXiv)

Self-supervised learning has been widely used to obtain transferrable representations from unlabeled images. Especially, recent contrastive learning methods have shown impressive performances on downstream image classification tasks. While these contrastive methods mainly focus on generating invariant global representations at the image-level under semantic-preserving transformations, they are prone to overlook spatial consistency of local representations and therefore have a limitation in pretraining for localization tasks such as object detection and instance segmentation. Moreover, aggressively cropped views used in existing contrastive methods can minimize representation distances between the semantically different regions of a single image. In this paper, we propose a spatially consistent representation learning algorithm (SCRL) for multi-object and location-specific tasks. In particular, we devise a novel self-supervised objective that tries to produce coherent spatial representations of a randomly cropped local region according to geometric translations and zooming operations. On various downstream localization tasks with benchmark datasets, the proposed SCRL shows significant performance improvements over the image-level supervised pretraining as well as the state-of-the-art self-supervised learning methods. Code is available at https://github.com/kakaobrain/scrl