# Rank-based Decomposable Losses in Machine Learning: A Survey

arXiv: https://arxiv.org/abs/2207.08768 · HTML: https://mlsanity.com/paper/2207.08768/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 89 of its 231 references that are in the map.
- First posted on arXiv 18 Jul 2022.

## Builds on (most cited first, 30 of 89)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels](https://mlsanity.com/paper/1804.06872/) (2018)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [Delving into Transferable Adversarial Examples and Black-box Attacks](https://mlsanity.com/paper/1611.02770/) (2016)
- [Deep Learning on a Data Diet: Finding Important Examples Early in Training](https://mlsanity.com/paper/2107.07075/) (2021)
- [Agnostic Federated Learning](https://mlsanity.com/paper/1902.00146/) (2019)
- [Not All Samples Are Created Equal: Deep Learning with Importance Sampling](https://mlsanity.com/paper/1803.00942/) (2018)
- [A survey of loss functions for semantic segmentation](https://mlsanity.com/paper/2006.14822/) (2020)
- [Coresets via Bilevel Optimization for Continual Learning and Streaming](https://mlsanity.com/paper/2006.03875/) (2020)
- [Large-Scale Methods for Distributionally Robust Optimization](https://mlsanity.com/paper/2010.05893/) (2020)
- [Variance-based regularization with convex objectives](https://mlsanity.com/paper/1610.02581/) (2016)
- [Learning with Fenchel-Young Losses](https://mlsanity.com/paper/1901.02324/) (2019)
- [Online Coreset Selection for Rehearsal-based Continual Learning](https://mlsanity.com/paper/2106.01085/) (2021)
- [Tilted Empirical Risk Minimization](https://mlsanity.com/paper/2007.01162/) (2020)
- [When Do Curricula Work?](https://mlsanity.com/paper/2012.03107/) (2020)
- [AUC Maximization in the Era of Big Data and AI: A Survey](https://mlsanity.com/paper/2203.15046/) (2022)
- [Fairness risk measures](https://mlsanity.com/paper/1901.08665/) (2019)
- [Optimizing Rank-based Metrics with Blackbox Differentiation](https://mlsanity.com/paper/1912.03500/) (2019)
- [DORO: Distributional and Outlier Robust Optimization](https://mlsanity.com/paper/2106.06142/) (2021)

## Built on by (most cited first, 26 of 26)

- [Preserving Fairness Generalization in Deepfake Detection](https://mlsanity.com/paper/2402.17229/) (2024)
- [Improving Fairness in Deepfake Detection](https://mlsanity.com/paper/2306.16635/) (2023)
- [Robust CLIP-Based Detector for Exposing Diffusion Model-Generated Images](https://mlsanity.com/paper/2404.12908/) (2024)
- [Robust Light-Weight Facial Affective Behavior Recognition with CLIP](https://mlsanity.com/paper/2403.09915/) (2024)
- [Distributionally Robust Survival Analysis: A Novel Fairness Loss Without Demographics](https://mlsanity.com/paper/2211.10508/) (2022)
- [UU-Mamba: Uncertainty-aware U-Mamba for Cardiac Image Segmentation](https://mlsanity.com/paper/2405.17496/) (2024)
- [Fairness in Survival Analysis with Distributionally Robust Optimization](https://mlsanity.com/paper/2409.10538/) (2024)
- [Robust COVID-19 Detection in CT Images with CLIP](https://mlsanity.com/paper/2403.08947/) (2024)
- [Robustly Optimized Deep Feature Decoupling Network for Fatty Liver Diseases Detection](https://mlsanity.com/paper/2406.17338/) (2024)
- [Outlier Robust Adversarial Training](https://mlsanity.com/paper/2309.05145/) (2023)
- [UMedNeRF: Uncertainty-aware Single View Volumetric Rendering for Medical Neural Radiance Fields](https://mlsanity.com/paper/2311.05836/) (2023)
- [Uncertainty-Aware Explainable Recommendation with Large Language Models](https://mlsanity.com/paper/2402.03366/) (2024)
- [Masked Conditional Diffusion Model for Enhancing Deepfake Detection](https://mlsanity.com/paper/2402.00541/) (2024)
- [Harnessing the Power of Text-image Contrastive Models for Automatic Detection of Online Misinformation](https://mlsanity.com/paper/2304.10249/) (2023)
- [Improving Generalization of Medical Image Registration Foundation Model](https://mlsanity.com/paper/2505.06527/) (2025)
- [Robust AI-Generated Face Detection with Imbalanced Data](https://mlsanity.com/paper/2505.02182/) (2025)
- [Robust Fairness Vision-Language Learning for Medical Image Analysis](https://mlsanity.com/paper/2505.03153/) (2025)
- [Attacking Important Pixels for Anchor-free Detectors](https://mlsanity.com/paper/2301.11457/) (2023)
- [Soft ascent-descent as a stable and flexible alternative to flooding](https://mlsanity.com/paper/2310.10006/) (2023)
- [An Uncertainty-Aware Generalization Framework for Cardiovascular Image Segmentation](https://mlsanity.com/paper/2409.14305/) (2024)
- [QAMRO: Quality-aware Adaptive Margin Ranking Optimization for Human-aligned Assessment of Audio Generation Systems](https://mlsanity.com/paper/2508.08957/) (2025)
- [Decision-Aligned Evaluation of Uncertainty Quantification](https://mlsanity.com/paper/2606.26990/) (2026)
- [Robust variance-regularized risk minimization with concomitant scaling](https://mlsanity.com/paper/2301.11584/) (2023)
- [Criterion Collapse and Loss Distribution Control](https://mlsanity.com/paper/2402.09802/) (2024)
- [Robust Multi-Source Covid-19 Detection in CT Images](https://mlsanity.com/paper/2604.03320/) (2026)
- [Robust Fair Disease Diagnosis in CT Images](https://mlsanity.com/paper/2604.09710/) (2026)

## Abstract (arXiv)

Recent works have revealed an essential paradigm in designing loss functions that differentiate individual losses vs. aggregate losses. The individual loss measures the quality of the model on a sample, while the aggregate loss combines individual losses/scores over each training sample. Both have a common procedure that aggregates a set of individual values to a single numerical value. The ranking order reflects the most fundamental relation among individual values in designing losses. In addition, decomposability, in which a loss can be decomposed into an ensemble of individual terms, becomes a significant property of organizing losses/scores. This survey provides a systematic and comprehensive review of rank-based decomposable losses in machine learning. Specifically, we provide a new taxonomy of loss functions that follows the perspectives of aggregate loss and individual loss. We identify the aggregator to form such losses, which are examples of set functions. We organize the rank-based decomposable losses into eight categories. Following these categories, we review the literature on rank-based aggregate losses and rank-based individual losses. We describe general formulas for these losses and connect them with existing research topics. We also suggest future research directions spanning unexplored, remaining, and emerging issues in rank-based decomposable losses.