# Reliable Conformal Prediction for Ordinal Classification Using the Ranked Probability Score

arXiv: https://arxiv.org/abs/2606.24959 · HTML: https://mlsanity.com/paper/2606.24959/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 74 references that are in the map.
- First posted on arXiv 23 Jun 2026.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification](https://mlsanity.com/paper/2107.07511/) (2021)
- [MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classification](https://mlsanity.com/paper/2110.14795/) (2021)
- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [Tabular Data: Deep Learning is Not All You Need](https://mlsanity.com/paper/2106.03253/) (2021)
- [Classification with Valid and Adaptive Coverage](https://mlsanity.com/paper/2006.02544/) (2020)
- [Conformal Risk Control](https://mlsanity.com/paper/2208.02814/) (2022)
- [Least Ambiguous Set-Valued Classifiers with Bounded Error Levels](https://mlsanity.com/paper/1609.00451/) (2016)
- [Rank consistent ordinal regression for neural networks with application to age estimation](https://mlsanity.com/paper/1901.07884/) (2019)
- [Deep Neural Networks for Rank-Consistent Ordinal Regression Based On Conditional Probabilities](https://mlsanity.com/paper/2111.08851/) (2021)
- [Squared Earth Mover's Distance-based Loss for Training Deep Neural Networks](https://mlsanity.com/paper/1611.05916/) (2016)
- [Improving Trustworthiness of AI Disease Severity Rating in Medical Imaging with Ordinal Conformal Prediction Sets](https://mlsanity.com/paper/2207.02238/) (2022)
- [Unimodal probability distributions for deep ordinal classification](https://mlsanity.com/paper/1705.05278/) (2017)
- [Unimodal Distributions for Ordinal Regression](https://mlsanity.com/paper/2303.04547/) (2023)
- [Class Distance Weighted Cross Entropy Loss for Classification of Disease Severity](https://mlsanity.com/paper/2412.01246/) (2024)
- [Performance Metrics for Probabilistic Ordinal Classifiers](https://mlsanity.com/paper/2309.08701/) (2023)
- [dlordinal: a Python package for deep ordinal classification](https://mlsanity.com/paper/2407.17163/) (2024)
- [Optimal Conformal Prediction under Epistemic Uncertainty](https://mlsanity.com/paper/2505.19033/) (2025)
- [Aleatoric and Epistemic Uncertainty Measures for Ordinal Classification through Binary Reduction](https://mlsanity.com/paper/2507.00733/) (2025)
- [TOC-UCO: a comprehensive repository of tabular ordinal classification datasets](https://mlsanity.com/paper/2507.17348/) (2025)

## Built on by (most cited first, 1 of 1)

- [CHOIR: heterogeneity-aware conformal prediction for crash injury severity across driver safety strata](https://mlsanity.com/paper/2609.11592/) (2026)

## Abstract (arXiv)

Ordinal classification (OC) arises in high-stakes domains such as medicine and finance, where uncertainty quantification must account for the severity of ordinal errors. Conformal prediction (CP) provides distribution-free prediction sets with marginal coverage guarantees; however, its practical effectiveness depends critically on the choice of nonconformity function. We introduce a CP method for ordinal classification based on the ranked probability score (RPS), a proper scoring rule defined over cumulative predictive distributions. Although it reflects ordinal risk quite naturally, it has largely been neglected in conformal ordinal prediction (COP). When used as a measure of nonconformity, RPS yields median-centered contiguous prediction sets by construction. The method is model-agnostic, supports both assessed and grouped ordered categorical outcomes, and permits efficient implementation compared to greedy interval selection procedures. Across multiple ordinal image and tabular datasets, RPS-based CP produces contiguous prediction sets and strikes a favorable balance between prediction set width and the magnitude of ordinal miscoverage relative to existing CP methods.