# How Nonconformity Functions and Difficulty of Datasets Impact the Efficiency of Conformal Classifiers

arXiv: https://arxiv.org/abs/2108.05677 · HTML: https://mlsanity.com/paper/2108.05677/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 17 references that are in the map.
- First posted on arXiv 12 Aug 2021.

## Builds on (most cited first, 1 of 1)

- [Selection and Aggregation of Conformal Prediction Sets](https://mlsanity.com/paper/2104.13871/) (2021)

## Built on by (most cited first, 1 of 1)

- [The Penalized Inverse Probability Measure for Conformal Classification](https://mlsanity.com/paper/2406.08884/) (2024)

## Abstract (arXiv)

The property of conformal predictors to guarantee the required accuracy rate makes this framework attractive in various practical applications. However, this property is achieved at a price of reduction in precision. In the case of conformal classification, the systems can output multiple class labels instead of one. It is also known from the literature, that the choice of nonconformity function has a major impact on the efficiency of conformal classifiers. Recently, it was shown that different model-agnostic nonconformity functions result in conformal classifiers with different characteristics. For a Neural Network-based conformal classifier, the inverse probability (or hinge loss) allows minimizing the average number of predicted labels, and margin results in a larger fraction of singleton predictions. In this work, we aim to further extend this study. We perform an experimental evaluation using 8 different classification algorithms and discuss when the previously observed relationship holds or not. Additionally, we propose a successful method to combine the properties of these two nonconformity functions. The experimental evaluation is done using 11 real and 5 synthetic datasets.