# Object-Aware Cropping for Self-Supervised Learning

arXiv: https://arxiv.org/abs/2112.00319 · HTML: https://mlsanity.com/paper/2112.00319/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 69 references that are in the map.
- First posted on arXiv 1 Dec 2021.

## Builds on (most cited first, 30 of 58)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [Self-Supervised Learning of Pretext-Invariant Representations](https://mlsanity.com/paper/1912.01991/) (2019)
- [AutoAugment: Learning Augmentation Policies from Data](https://mlsanity.com/paper/1805.09501/) (2018)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [Prototypical Contrastive Learning of Unsupervised Representations](https://mlsanity.com/paper/2005.04966/) (2020)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [A Theoretical Analysis of Contrastive Unsupervised Representation Learning](https://mlsanity.com/paper/1902.09229/) (2019)
- [Dense Contrastive Learning for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2011.09157/) (2020)
- [Hard Negative Mixing for Contrastive Learning](https://mlsanity.com/paper/2010.01028/) (2020)

## Built on by (most cited first, 22 of 22)

- [Object discovery and representation networks](https://mlsanity.com/paper/2203.08777/) (2022)
- [Crafting Better Contrastive Views for Siamese Representation Learning](https://mlsanity.com/paper/2202.03278/) (2022)
- [Hyperbolic Contrastive Learning for Visual Representations beyond Objects](https://mlsanity.com/paper/2212.00653/) (2022)
- [Know Your Self-supervised Learning: A Survey on Image-based Generative and Discriminative Training](https://mlsanity.com/paper/2305.13689/) (2023)
- [Unsupervised Dense Retrieval with Relevance-Aware Contrastive Pre-Training](https://mlsanity.com/paper/2306.03166/) (2023)
- [Learning Visual Representations via Language-Guided Sampling](https://mlsanity.com/paper/2302.12248/) (2023)
- [Understanding Hyperbolic Metric Learning through Hard Negative Sampling](https://mlsanity.com/paper/2404.15523/) (2024)
- [Understanding Self-Supervised Pretraining with Part-Aware Representation Learning](https://mlsanity.com/paper/2301.11915/) (2023)
- [Coarse Is Better? A New Pipeline Towards Self-Supervised Learning with Uncurated Images](https://mlsanity.com/paper/2306.04244/) (2023)
- [Self-Supervised Pyramid Representation Learning for Multi-Label Visual Analysis and Beyond](https://mlsanity.com/paper/2208.14439/) (2022)
- [LeOCLR: Leveraging Original Images for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2403.06813/) (2024)
- [Saliency Guided Contrastive Learning on Scene Images](https://mlsanity.com/paper/2302.11461/) (2023)
- [Learning Self-Regularized Adversarial Views for Self-Supervised Vision Transformers](https://mlsanity.com/paper/2210.08458/) (2022)
- [A Contrastive Learning Foundation Model Based on Perfectly Aligned Sample Pairs for Remote Sensing Images](https://mlsanity.com/paper/2505.19447/) (2025)
- [Beyond Instance Consistency: Investigating View Diversity in Self-supervised Learning](https://mlsanity.com/paper/2509.11344/) (2025)
- [Focusing On Targets For Improving Weakly Supervised Visual Grounding](https://mlsanity.com/paper/2302.11252/) (2023)
- [The Bad Batches: Enhancing Self-Supervised Learning in Image Classification Through Representative Batch Curation](https://mlsanity.com/paper/2403.19579/) (2024)
- [Taming the Randomness: Towards Label-Preserving Cropping in Contrastive Learning](https://mlsanity.com/paper/2504.19824/) (2025)
- [Multiple Object Stitching for Unsupervised Representation Learning](https://mlsanity.com/paper/2506.07364/) (2025)
- [Self-Supervised Learning with a Multi-Task Latent Space Objective](https://mlsanity.com/paper/2602.05845/) (2026)
- [ChimeraLoRA: Multi-Head LoRA-Guided Synthetic Datasets](https://mlsanity.com/paper/2602.19708/) (2026)
- [Mask-supervised Object-centric Representation Learning with LeJEPA](https://mlsanity.com/paper/2607.02404/) (2026)

## Abstract (arXiv)

A core component of the recent success of self-supervised learning is cropping data augmentation, which selects sub-regions of an image to be used as positive views in the self-supervised loss. The underlying assumption is that randomly cropped and resized regions of a given image share information about the objects of interest, which the learned representation will capture. This assumption is mostly satisfied in datasets such as ImageNet where there is a large, centered object, which is highly likely to be present in random crops of the full image. However, in other datasets such as OpenImages or COCO, which are more representative of real world uncurated data, there are typically multiple small objects in an image. In this work, we show that self-supervised learning based on the usual random cropping performs poorly on such datasets. We propose replacing one or both of the random crops with crops obtained from an object proposal algorithm. This encourages the model to learn both object and scene level semantic representations. Using this approach, which we call object-aware cropping, results in significant improvements over scene cropping on classification and object detection benchmarks. For example, on OpenImages, our approach achieves an improvement of 8.8% mAP over random scene-level cropping using MoCo-v2 based pre-training. We also show significant improvements on COCO and PASCAL-VOC object detection and segmentation tasks over the state-of-the-art self-supervised learning approaches. Our approach is efficient, simple and general, and can be used in most existing contrastive and non-contrastive self-supervised learning frameworks.