# Information-guided pixel augmentation for pixel-wise contrastive learning

arXiv: https://arxiv.org/abs/2211.07118 · HTML: https://mlsanity.com/paper/2211.07118/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 57 references that are in the map.
- First posted on arXiv 14 Nov 2022.

## Builds on (most cited first, 30 of 44)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [Contrastive Learning with Hard Negative Samples](https://mlsanity.com/paper/2010.04592/) (2020)
- [Hard Negative Mixing for Contrastive Learning](https://mlsanity.com/paper/2010.01028/) (2020)
- [Debiased Contrastive Learning](https://mlsanity.com/paper/2007.00224/) (2020)
- [Contrastive learning of global and local features for medical image segmentation with limited annotations](https://mlsanity.com/paper/2006.10511/) (2020)
- [Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/2011.10043/) (2020)
- [Exploring Cross-Image Pixel Contrast for Semantic Segmentation](https://mlsanity.com/paper/2101.11939/) (2021)
- [Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals](https://mlsanity.com/paper/2102.06191/) (2021)
- [Semi-Supervised Semantic Segmentation with Pixel-Level Contrastive Learning from a Class-wise Memory Bank](https://mlsanity.com/paper/2104.13415/) (2021)
- [Pixel Contrastive-Consistent Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2108.09025/) (2021)
- [Contrastive Learning with Stronger Augmentations](https://mlsanity.com/paper/2104.07713/) (2021)
- [Contrastive Learning for Label-Efficient Semantic Segmentation](https://mlsanity.com/paper/2012.06985/) (2020)
- [When Does Contrastive Visual Representation Learning Work?](https://mlsanity.com/paper/2105.05837/) (2021)
- [Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast](https://mlsanity.com/paper/2110.07110/) (2021)
- [Bootstrapping Semantic Segmentation with Regional Contrast](https://mlsanity.com/paper/2104.04465/) (2021)
- [Integrating Spatial Configuration into Heatmap Regression Based CNNs for Landmark Localization](https://mlsanity.com/paper/1908.00748/) (2019)
- [SAM: Self-supervised Learning of Pixel-wise Anatomical Embeddings in Radiological Images](https://mlsanity.com/paper/2012.02383/) (2020)
- [Un-Mix: Rethinking Image Mixtures for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/2003.05438/) (2020)
- [MixCo: Mix-up Contrastive Learning for Visual Representation](https://mlsanity.com/paper/2010.06300/) (2020)

## Built on by (most cited first, 4 of 4)

- [Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspective](https://mlsanity.com/paper/2302.01735/) (2023)
- [ACTION++: Improving Semi-supervised Medical Image Segmentation with Adaptive Anatomical Contrast](https://mlsanity.com/paper/2304.02689/) (2023)
- [Towards multi-modal anatomical landmark detection for ultrasound-guided brain tumor resection with contrastive learning](https://mlsanity.com/paper/2307.14523/) (2023)
- [GCL: Gradient-Guided Contrastive Learning for Medical Image Segmentation with Multi-Perspective Meta Labels](https://mlsanity.com/paper/2309.08888/) (2023)

## Abstract (arXiv)

Contrastive learning (CL) is a form of self-supervised learning and has been widely used for various tasks. Different from widely studied instance-level contrastive learning, pixel-wise contrastive learning mainly helps with pixel-wise tasks such as medical landmark detection. The counterpart to an instance in instance-level CL is a pixel, along with its neighboring context, in pixel-wise CL. Aiming to build better feature representation, there is a vast literature about designing instance augmentation strategies for instance-level CL; but there is little similar work on pixel augmentation for pixel-wise CL with a pixel granularity. In this paper, we attempt to bridge this gap. We first classify a pixel into three categories, namely low-, medium-, and high-informative, based on the information quantity the pixel contains. Inspired by the ``InfoMin" principle, we then design separate augmentation strategies for each category in terms of augmentation intensity and sampling ratio. Extensive experiments validate that our information-guided pixel augmentation strategy succeeds in encoding more discriminative representations and surpassing other competitive approaches in unsupervised local feature matching. Furthermore, our pretrained model improves the performance of both one-shot and fully supervised models. To the best of our knowledge, we are the first to propose a pixel augmentation method with a pixel granularity for enhancing unsupervised pixel-wise contrastive learning.