# Conditional Alignment and Uniformity for Contrastive Learning with Continuous Proxy Labels

arXiv: https://arxiv.org/abs/2111.05643 · HTML: https://mlsanity.com/paper/2111.05643/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 35 references that are in the map.
- First posted on arXiv 10 Nov 2021.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [A Theoretical Analysis of Contrastive Unsupervised Representation Learning](https://mlsanity.com/paper/1902.09229/) (2019)
- [Transfusion: Understanding Transfer Learning for Medical Imaging](https://mlsanity.com/paper/1902.07208/) (2019)
- [Debiased Contrastive Learning](https://mlsanity.com/paper/2007.00224/) (2020)
- [Big Self-Supervised Models Advance Medical Image Classification](https://mlsanity.com/paper/2101.05224/) (2021)
- [Contrastive learning of global and local features for medical image segmentation with limited annotations](https://mlsanity.com/paper/2006.10511/) (2020)
- [Colorization as a Proxy Task for Visual Understanding](https://mlsanity.com/paper/1703.04044/) (2017)
- [3D Self-Supervised Methods for Medical Imaging](https://mlsanity.com/paper/2006.03829/) (2020)
- [Convolutional Neural Networks for Classification of Alzheimer's Disease: Overview and Reproducible Evaluation](https://mlsanity.com/paper/1904.07773/) (2019)
- [EnD: Entangling and Disentangling deep representations for bias correction](https://mlsanity.com/paper/2103.02023/) (2021)
- [Contrastive Learning with Continuous Proxy Meta-Data for 3D MRI Classification](https://mlsanity.com/paper/2106.08808/) (2021)
- [Deep Learning Fundus Image Analysis for Diabetic Retinopathy and Macular Edema Grading](https://mlsanity.com/paper/1904.08764/) (2019)

## Built on by (most cited first, 7 of 7)

- [Rank-N-Contrast: Learning Continuous Representations for Regression](https://mlsanity.com/paper/2210.01189/) (2022)
- [Separating common from salient patterns with Contrastive Representation Learning](https://mlsanity.com/paper/2402.11928/) (2024)
- [Graph-based Alignment and Uniformity for Recommendation](https://mlsanity.com/paper/2308.09292/) (2023)
- [SupReMix: Supervised Contrastive Learning for Medical Imaging Regression with Mixup](https://mlsanity.com/paper/2309.16633/) (2023)
- [Decoupled conditional contrastive learning with variable metadata for prostate lesion detection](https://mlsanity.com/paper/2308.09542/) (2023)
- [Automatic Discovery of Disease Subgroups by Contrasting with Healthy Controls](https://mlsanity.com/paper/2605.21301/) (2026)
- [Robust brain age estimation from structural MRI with contrastive learning](https://mlsanity.com/paper/2507.01794/) (2025)

## Abstract (arXiv)

Contrastive Learning has shown impressive results on natural and medical images, without requiring annotated data. However, a particularity of medical images is the availability of meta-data (such as age or sex) that can be exploited for learning representations. Here, we show that the recently proposed contrastive y-Aware InfoNCE loss, that integrates multi-dimensional meta-data, asymptotically optimizes two properties: conditional alignment and global uniformity. Similarly to [Wang, 2020], conditional alignment means that similar samples should have similar features, but conditionally on the meta-data. Instead, global uniformity means that the (normalized) features should be uniformly distributed on the unit hyper-sphere, independently of the meta-data. Here, we propose to define conditional uniformity, relying on the meta-data, that repel only samples with dissimilar meta-data. We show that direct optimization of both conditional alignment and uniformity improves the representations, in terms of linear evaluation, on both CIFAR-100 and a brain MRI dataset.