# Trustworthy Classification through Rank-Based Conformal Prediction Sets

arXiv: https://arxiv.org/abs/2407.04407 · HTML: https://mlsanity.com/paper/2407.04407/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 38 references that are in the map.
- First posted on arXiv 5 Jul 2024.

## Builds on (most cited first, 28 of 28)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Conformal Prediction Under Covariate Shift](https://mlsanity.com/paper/1904.06019/) (2019)
- [Classification with Valid and Adaptive Coverage](https://mlsanity.com/paper/2006.02544/) (2020)
- [Uncertainty Sets for Image Classifiers using Conformal Prediction](https://mlsanity.com/paper/2009.14193/) (2020)
- [Conformal Risk Control](https://mlsanity.com/paper/2208.02814/) (2022)
- [Least Ambiguous Set-Valued Classifiers with Bounded Error Levels](https://mlsanity.com/paper/1609.00451/) (2016)
- [Conformal Language Modeling](https://mlsanity.com/paper/2306.10193/) (2023)
- [Conformal Prediction with Large Language Models for Multi-Choice Question Answering](https://mlsanity.com/paper/2305.18404/) (2023)
- [Testing for Outliers with Conformal p-values](https://mlsanity.com/paper/2104.08279/) (2021)
- [Learning Optimal Conformal Classifiers](https://mlsanity.com/paper/2110.09192/) (2021)
- [Distribution-free uncertainty quantification for classification under label shift](https://mlsanity.com/paper/2103.03323/) (2021)
- [Uncertainty Quantification over Graph with Conformalized Graph Neural Networks](https://mlsanity.com/paper/2305.14535/) (2023)
- [Conformal prediction for time series](https://mlsanity.com/paper/2010.09107/) (2020)
- [Conformal Prediction for Deep Classifier via Label Ranking](https://mlsanity.com/paper/2310.06430/) (2023)
- [Conformal Prediction for Time Series with Modern Hopfield Networks](https://mlsanity.com/paper/2303.12783/) (2023)
- [Anomalous Edge Detection in Edge Exchangeable Social Network Models](https://mlsanity.com/paper/2109.12727/) (2021)
- [Conformal Autoregressive Generation: Beam Search with Coverage Guarantees](https://mlsanity.com/paper/2309.03797/) (2023)
- [Conformal Prediction for Network-Assisted Regression](https://mlsanity.com/paper/2302.10095/) (2023)
- [Conformal Load Prediction with Transductive Graph Autoencoders](https://mlsanity.com/paper/2406.08281/) (2024)
- [Conformal link prediction for false discovery rate control](https://mlsanity.com/paper/2306.14693/) (2023)
- [Conformalized Link Prediction on Graph Neural Networks](https://mlsanity.com/paper/2406.18763/) (2024)
- [Multi-label Classification under Uncertainty: A Tree-based Conformal Prediction Approach](https://mlsanity.com/paper/2404.19472/) (2024)

## Built on by (most cited first, 20 of 20)

- [Weighted Aggregation of Conformity Scores for Classification](https://mlsanity.com/paper/2407.10230/) (2024)
- [Conformal Thresholded Intervals for Efficient Regression](https://mlsanity.com/paper/2407.14495/) (2024)
- [Game-Theoretic Defenses for Robust Conformal Prediction Against Adversarial Attacks in Medical Imaging](https://mlsanity.com/paper/2411.04376/) (2024)
- [Entropy Reweighted Conformal Classification](https://mlsanity.com/paper/2407.17377/) (2024)
- [Conformalized Interval Arithmetic with Symmetric Calibration](https://mlsanity.com/paper/2408.10939/) (2024)
- [C-Adapter: Adapting Deep Classifiers for Efficient Conformal Prediction Sets](https://mlsanity.com/paper/2410.09408/) (2024)
- [Conditional Conformal Risk Adaptation](https://mlsanity.com/paper/2504.07611/) (2025)
- [Enhancing Trustworthiness of Graph Neural Networks with Rank-Based Conformal Training](https://mlsanity.com/paper/2501.02767/) (2025)
- [Volume-Sorted Prediction Set: Efficient Conformal Prediction for Multi-Target Regression](https://mlsanity.com/paper/2503.02205/) (2025)
- [Enhancing Adversarial Robustness with Conformal Prediction: A Framework for Guaranteed Model Reliability](https://mlsanity.com/paper/2506.07804/) (2025)
- [Enhanced Route Planning with Calibrated Uncertainty Set](https://mlsanity.com/paper/2503.10088/) (2025)
- [Adaptive Conformal Inference by Particle Filtering under Hidden Markov Models](https://mlsanity.com/paper/2411.01558/) (2024)
- [Symmetric Aggregation of Conformity Scores for Efficient Uncertainty Sets](https://mlsanity.com/paper/2512.06945/) (2025)
- [Uncertainty-Aware Trajectory Prediction via Rule-Regularized Heteroscedastic Deep Classification](https://mlsanity.com/paper/2504.13111/) (2025)
- [Conformal Object Detection by Sequential Risk Control](https://mlsanity.com/paper/2505.24038/) (2025)
- [Residual Reweighted Conformal Prediction for Graph Neural Networks](https://mlsanity.com/paper/2506.07854/) (2025)
- [Empirical Bayes Conformal Prediction for Vision and Language Models](https://mlsanity.com/paper/2605.23189/) (2026)
- [Softmax is not Enough (for Adaptive Conformal Classification)](https://mlsanity.com/paper/2602.19498/) (2026)
- [CRISP: Rank-Guided Iterative Squeezing for Robust Medical Image Segmentation under Domain Shift](https://mlsanity.com/paper/2604.05409/) (2026)
- [CRISP: Constrained Refinement via Iterative Squeezing Process for Robust Medical Image Segmentation under Domain Shift](https://mlsanity.com/paper/2607.15231/) (2026)

## Abstract (arXiv)

Machine learning classification tasks often benefit from predicting a set of possible labels with confidence scores to capture uncertainty. However, existing methods struggle with the high-dimensional nature of the data and the lack of well-calibrated probabilities from modern classification models. We propose a novel conformal prediction method that employs a rank-based score function suitable for classification models that predict the order of labels correctly, even if not well-calibrated. Our approach constructs prediction sets that achieve the desired coverage rate while managing their size. We provide a theoretical analysis of the expected size of the conformal prediction sets based on the rank distribution of the underlying classifier. Through extensive experiments, we demonstrate that our method outperforms existing techniques on various datasets, providing reliable uncertainty quantification. Our contributions include a novel conformal prediction method, theoretical analysis, and empirical evaluation. This work advances the practical deployment of machine learning systems by enabling reliable uncertainty quantification.