# RankSim: Ranking Similarity Regularization for Deep Imbalanced Regression

arXiv: https://arxiv.org/abs/2205.15236 · HTML: https://mlsanity.com/paper/2205.15236/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 40 references that are in the map.
- First posted on arXiv 30 May 2022.

## Builds on (most cited first, 28 of 28)

- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation](https://mlsanity.com/paper/1708.00055/) (2017)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [Decoupling Representation and Classifier for Long-Tailed Recognition](https://mlsanity.com/paper/1910.09217/) (2019)
- [Large-Scale Long-Tailed Recognition in an Open World](https://mlsanity.com/paper/1904.05160/) (2019)
- [A systematic study of the class imbalance problem in convolutional neural networks](https://mlsanity.com/paper/1710.05381/) (2017)
- [Balanced Meta-Softmax for Long-Tailed Visual Recognition](https://mlsanity.com/paper/2007.10740/) (2020)
- [Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect](https://mlsanity.com/paper/2009.12991/) (2020)
- [What is the Effect of Importance Weighting in Deep Learning?](https://mlsanity.com/paper/1812.03372/) (2018)
- [Differentiation of Blackbox Combinatorial Solvers](https://mlsanity.com/paper/1912.02175/) (2019)
- [Rethinking the Value of Labels for Improving Class-Imbalanced Learning](https://mlsanity.com/paper/2006.07529/) (2020)
- [Improving Calibration for Long-Tailed Recognition](https://mlsanity.com/paper/2104.00466/) (2021)
- [Learning with Average Precision: Training Image Retrieval with a Listwise Loss](https://mlsanity.com/paper/1906.07589/) (2019)
- [Delving into Deep Imbalanced Regression](https://mlsanity.com/paper/2102.09554/) (2021)
- [Remix: Rebalanced Mixup](https://mlsanity.com/paper/2007.03943/) (2020)
- [Differentiable Ranks and Sorting using Optimal Transport](https://mlsanity.com/paper/1905.11885/) (2019)
- [Implicit Semantic Data Augmentation for Deep Networks](https://mlsanity.com/paper/1909.12220/) (2019)
- [Optimizing Rank-based Metrics with Blackbox Differentiation](https://mlsanity.com/paper/1912.03500/) (2019)
- [Posterior Re-calibration for Imbalanced Datasets](https://mlsanity.com/paper/2010.11820/) (2020)
- [Learning Deep Embeddings with Histogram Loss](https://mlsanity.com/paper/1611.00822/) (2016)
- [Self-Damaging Contrastive Learning](https://mlsanity.com/paper/2106.02990/) (2021)
- [Towards Calibrated Model for Long-Tailed Visual Recognition from Prior Perspective](https://mlsanity.com/paper/2111.03874/) (2021)
- [PML: Progressive Margin Loss for Long-tailed Age Classification](https://mlsanity.com/paper/2103.02140/) (2021)
- [SoDeep: a Sorting Deep net to learn ranking loss surrogates](https://mlsanity.com/paper/1904.04272/) (2019)
- [Image-Level or Object-Level? A Tale of Two Resampling Strategies for Long-Tailed Detection](https://mlsanity.com/paper/2104.05702/) (2021)
- [Training Deep Neural Networks via Direct Loss Minimization](https://mlsanity.com/paper/1511.06411/) (2015)
- [Hashing as Tie-Aware Learning to Rank](https://mlsanity.com/paper/1705.08562/) (2017)

## Built on by (most cited first, 30 of 52)

- [Rank-N-Contrast: Learning Continuous Representations for Regression](https://mlsanity.com/paper/2210.01189/) (2022)
- [Improving Deep Regression with Ordinal Entropy](https://mlsanity.com/paper/2301.08915/) (2023)
- [Learning-to-Rank Meets Language: Boosting Language-Driven Ordering Alignment for Ordinal Classification](https://mlsanity.com/paper/2306.13856/) (2023)
- [ConR: Contrastive Regularizer for Deep Imbalanced Regression](https://mlsanity.com/paper/2309.06651/) (2023)
- [A step towards understanding why classification helps regression](https://mlsanity.com/paper/2308.10603/) (2023)
- [Semi-Supervised Graph Imbalanced Regression](https://mlsanity.com/paper/2305.12087/) (2023)
- [AFreeCA: Annotation-Free Counting for All](https://mlsanity.com/paper/2403.04943/) (2024)
- [CLOC: Contrastive Learning for Ordinal Classification with Multi-Margin N-pair Loss](https://mlsanity.com/paper/2504.17813/) (2025)
- [A Comprehensive Survey on Imbalanced Data Learning](https://mlsanity.com/paper/2502.08960/) (2025)
- [Deep Imbalanced Regression via Hierarchical Classification Adjustment](https://mlsanity.com/paper/2310.17154/) (2023)
- [IM-Context: In-Context Learning for Imbalanced Regression Tasks](https://mlsanity.com/paper/2405.18202/) (2024)
- [Interpretable Cascading Mixture-of-Experts for Urban Traffic Congestion Prediction](https://mlsanity.com/paper/2406.12923/) (2024)
- [Rank and Align: Towards Effective Source-free Graph Domain Adaptation](https://mlsanity.com/paper/2408.12185/) (2024)
- [Improve Representation for Imbalanced Regression through Geometric Constraints](https://mlsanity.com/paper/2503.00876/) (2025)
- [Are we making much progress? Revisiting chemical reaction yield prediction from an imbalanced regression perspective](https://mlsanity.com/paper/2402.05971/) (2024)
- [Data Augmentation with Variational Autoencoder for Imbalanced Dataset](https://mlsanity.com/paper/2412.07039/) (2024)
- [Uncertainty Voting Ensemble for Imbalanced Deep Regression](https://mlsanity.com/paper/2305.15178/) (2023)
- [Continual Learning in Predictive Autoscaling](https://mlsanity.com/paper/2307.15941/) (2023)
- [Generalized Oversampling for Learning from Imbalanced datasets and Associated Theory](https://mlsanity.com/paper/2308.02966/) (2023)
- [Sample Selection via Contrastive Fragmentation for Noisy Label Regression](https://mlsanity.com/paper/2502.17771/) (2025)
- [Dist Loss: Enhancing Regression in Few-Shot Region through Distribution Distance Constraint](https://mlsanity.com/paper/2411.15216/) (2024)
- [AdaFlood: Adaptive Flood Regularization](https://mlsanity.com/paper/2311.02891/) (2023)
- [Gradient Aligned Regression via Pairwise Losses](https://mlsanity.com/paper/2402.06104/) (2024)
- [Boarding for ISS: Imbalanced Self-Supervised: Discovery of a Scaled Autoencoder for Mixed Tabular Datasets](https://mlsanity.com/paper/2403.15790/) (2024)
- [PoGDiff: Product-of-Gaussians Diffusion Models for Imbalanced Text-to-Image Generation](https://mlsanity.com/paper/2502.08106/) (2025)
- [Improving Deep Regression with Tightness](https://mlsanity.com/paper/2502.09122/) (2025)
- [Ranking Regularization for Critical Rare Classes: Minimizing False Positives at a High True Positive Rate](https://mlsanity.com/paper/2304.00049/) (2023)
- [Variational Imbalanced Regression: Fair Uncertainty Quantification via Probabilistic Smoothing](https://mlsanity.com/paper/2306.06599/) (2023)
- [Optimizing Automatic Speech Assessment: W-RankSim Regularization and Hybrid Feature Fusion Strategies](https://mlsanity.com/paper/2406.10873/) (2024)
- [Deep Imbalanced Regression to Estimate Vascular Age from PPG Data: a Novel Digital Biomarker for Cardiovascular Health](https://mlsanity.com/paper/2406.14953/) (2024)

## Abstract (arXiv)

Data imbalance, in which a plurality of the data samples come from a small proportion of labels, poses a challenge in training deep neural networks. Unlike classification, in regression the labels are continuous, potentially boundless, and form a natural ordering. These distinct features of regression call for new techniques that leverage the additional information encoded in label-space relationships. This paper presents the RankSim (ranking similarity) regularizer for deep imbalanced regression, which encodes an inductive bias that samples that are closer in label space should also be closer in feature space. In contrast to recent distribution smoothing based approaches, RankSim captures both nearby and distant relationships: for a given data sample, RankSim encourages the sorted list of its neighbors in label space to match the sorted list of its neighbors in feature space. RankSim is complementary to conventional imbalanced learning techniques, including re-weighting, two-stage training, and distribution smoothing, and lifts the state-of-the-art performance on three imbalanced regression benchmarks: IMDB-WIKI-DIR, AgeDB-DIR, and STS-B-DIR.