# Rank-N-Contrast: Learning Continuous Representations for Regression

arXiv: https://arxiv.org/abs/2210.01189 · HTML: https://mlsanity.com/paper/2210.01189/

## Key facts

- Cited by 76 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 52 references that are in the map.
- First posted on arXiv 3 Oct 2022.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [Debiased Contrastive Learning](https://mlsanity.com/paper/2007.00224/) (2020)
- [Delving into Deep Imbalanced Regression](https://mlsanity.com/paper/2102.09554/) (2021)
- [Rank consistent ordinal regression for neural networks with application to age estimation](https://mlsanity.com/paper/1901.07884/) (2019)
- [Targeted Supervised Contrastive Learning for Long-Tailed Recognition](https://mlsanity.com/paper/2111.13998/) (2021)
- [It's Written All Over Your Face: Full-Face Appearance-Based Gaze Estimation](https://mlsanity.com/paper/1611.08860/) (2016)
- [MPIIGaze: Real-World Dataset and Deep Appearance-Based Gaze Estimation](https://mlsanity.com/paper/1711.09017/) (2017)
- [Learnable latent embeddings for joint behavioral and neural analysis](https://mlsanity.com/paper/2204.00673/) (2022)
- [Selective-Supervised Contrastive Learning with Noisy Labels](https://mlsanity.com/paper/2203.04181/) (2022)
- [Deep Label Distribution Learning with Label Ambiguity](https://mlsanity.com/paper/1611.01731/) (2016)
- [L2CS-Net: Fine-Grained Gaze Estimation in Unconstrained Environments](https://mlsanity.com/paper/2203.03339/) (2022)
- [Deep Neural Networks for Rank-Consistent Ordinal Regression Based On Conditional Probabilities](https://mlsanity.com/paper/2111.08851/) (2021)
- [Learning Probabilistic Ordinal Embeddings for Uncertainty-Aware Regression](https://mlsanity.com/paper/2103.13629/) (2021)
- [RankSim: Ranking Similarity Regularization for Deep Imbalanced Regression](https://mlsanity.com/paper/2205.15236/) (2022)
- [Contrastive Learning with Continuous Proxy Meta-Data for 3D MRI Classification](https://mlsanity.com/paper/2106.08808/) (2021)
- [Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learning](https://mlsanity.com/paper/2105.14289/) (2021)
- [On Multi-Domain Long-Tailed Recognition, Imbalanced Domain Generalization and Beyond](https://mlsanity.com/paper/2203.09513/) (2022)
- [Conditional Alignment and Uniformity for Contrastive Learning with Continuous Proxy Labels](https://mlsanity.com/paper/2111.05643/) (2021)
- [Visual Weather Temperature Prediction](https://mlsanity.com/paper/1801.08267/) (2018)

## Built on by (most cited first, 30 of 76)

- [ConR: Contrastive Regularizer for Deep Imbalanced Regression](https://mlsanity.com/paper/2309.06651/) (2023)
- [Anatomical Foundation Models for Brain MRIs](https://mlsanity.com/paper/2408.07079/) (2024)
- [TractGeoNet: A geometric deep learning framework for pointwise analysis of tract microstructure to predict language assessment performance](https://mlsanity.com/paper/2307.03982/) (2023)
- [CLOC: Contrastive Learning for Ordinal Classification with Multi-Margin N-pair Loss](https://mlsanity.com/paper/2504.17813/) (2025)
- [GT-Loc: Unifying When and Where in Images Through a Joint Embedding Space](https://mlsanity.com/paper/2507.10473/) (2025)
- [SurvRNC: Learning Ordered Representations for Survival Prediction using Rank-N-Contrast](https://mlsanity.com/paper/2403.10603/) (2024)
- [LG-Gaze: Learning Geometry-aware Continuous Prompts for Language-Guided Gaze Estimation](https://mlsanity.com/paper/2411.08606/) (2024)
- [CoReEcho: Continuous Representation Learning for 2D+time Echocardiography Analysis](https://mlsanity.com/paper/2403.10164/) (2024)
- [Improve Representation for Imbalanced Regression through Geometric Constraints](https://mlsanity.com/paper/2503.00876/) (2025)
- [Latent Expression Generation for Referring Image Segmentation and Grounding](https://mlsanity.com/paper/2508.05123/) (2025)
- [Geometric Transformation Uncertainty for Improving 3D Fetal Brain Pose Prediction from Freehand 2D Ultrasound Videos](https://mlsanity.com/paper/2405.13235/) (2024)
- [DINO-Detect: A Simple yet Effective Framework for Blur-Robust AI-Generated Image Detection](https://mlsanity.com/paper/2511.12511/) (2025)
- [Toward a Well-Calibrated Discrimination via Survival Outcome-Aware Contrastive Learning](https://mlsanity.com/paper/2410.11340/) (2024)
- [From Age Estimation to Age-Invariant Face Recognition: Generalized Age Feature Extraction Using Order-Enhanced Contrastive Learning](https://mlsanity.com/paper/2501.01760/) (2025)
- [Provable Ordering and Continuity in Vision-Language Pretraining for Generalizable Embodied Agents](https://mlsanity.com/paper/2502.01218/) (2025)
- [On the rankability of visual embeddings](https://mlsanity.com/paper/2507.03683/) (2025)
- [TrajSurv: Learning Continuous Latent Trajectories from Electronic Health Records for Trustworthy Survival Prediction](https://mlsanity.com/paper/2508.00657/) (2025)
- [SupReMix: Supervised Contrastive Learning for Medical Imaging Regression with Mixup](https://mlsanity.com/paper/2309.16633/) (2023)
- [Iterative Quantum Feature Maps](https://mlsanity.com/paper/2506.19461/) (2025)
- [Distance-informed Neural Processes](https://mlsanity.com/paper/2508.18903/) (2025)
- [A robust PPG foundation model using multimodal physiological supervision](https://mlsanity.com/paper/2606.07365/) (2026)
- [Gradient Aligned Regression via Pairwise Losses](https://mlsanity.com/paper/2402.06104/) (2024)
- [Contrastive Learning for Implicit Social Factors in Social Media Popularity Prediction](https://mlsanity.com/paper/2410.09345/) (2024)
- [Enhancing Multimodal Sentiment Analysis for Missing Modality through Self-Distillation and Unified Modality Cross-Attention](https://mlsanity.com/paper/2410.15029/) (2024)
- [Alleviating Performance Disparity in Adversarial Spatiotemporal Graph Learning Under Zero-Inflated Distribution](https://mlsanity.com/paper/2504.00721/) (2025)
- [RankList -- A Listwise Preference Learning Framework for Predicting Subjective Preferences](https://mlsanity.com/paper/2508.09826/) (2025)
- [FoundDiff: Foundational Diffusion Model for Generalizable Low-Dose CT Denoising](https://mlsanity.com/paper/2508.17299/) (2025)
- [Leveraging semantic similarity for experimentation with AI-generated treatments](https://mlsanity.com/paper/2510.21119/) (2025)
- [Something from Nothing: Data Augmentation for Robust Severity Level Estimation of Dysarthric Speech](https://mlsanity.com/paper/2603.15988/) (2026)
- [CLDR: Contrastive Learning Drug Response Models from Natural Language Supervision](https://mlsanity.com/paper/2312.10707/) (2023)

## Abstract (arXiv)

Deep regression models typically learn in an end-to-end fashion without explicitly emphasizing a regression-aware representation. Consequently, the learned representations exhibit fragmentation and fail to capture the continuous nature of sample orders, inducing suboptimal results across a wide range of regression tasks. To fill the gap, we propose Rank-N-Contrast (RNC), a framework that learns continuous representations for regression by contrasting samples against each other based on their rankings in the target space. We demonstrate, theoretically and empirically, that RNC guarantees the desired order of learned representations in accordance with the target orders, enjoying not only better performance but also significantly improved robustness, efficiency, and generalization. Extensive experiments using five real-world regression datasets that span computer vision, human-computer interaction, and healthcare verify that RNC achieves state-of-the-art performance, highlighting its intriguing properties including better data efficiency, robustness to spurious targets and data corruptions, and generalization to distribution shifts. Code is available at: https://github.com/kaiwenzha/Rank-N-Contrast.