# Good Classification Measures and How to Find Them

arXiv: https://arxiv.org/abs/2201.09044 · HTML: https://mlsanity.com/paper/2201.09044/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 35 references that are in the map.
- First posted on arXiv 22 Jan 2022.

## Builds on (most cited first, 6 of 6)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation](https://mlsanity.com/paper/2010.16061/) (2020)
- [A note on the triangle inequality for the Jaccard distance](https://mlsanity.com/paper/1612.02696/) (2016)
- [Precipitation Nowcasting with Satellite Imagery](https://mlsanity.com/paper/1905.09932/) (2019)

## Built on by (most cited first, 11 of 11)

- [Characterizing Graph Datasets for Node Classification: Homophily-Heterophily Dichotomy and Beyond](https://mlsanity.com/paper/2209.06177/) (2022)
- [Why only Micro-F1? Class Weighting of Measures for Relation Classification](https://mlsanity.com/paper/2205.09460/) (2022)
- [Range-Based Equal Error Rate for Spoof Localization](https://mlsanity.com/paper/2305.17739/) (2023)
- [Revisiting Graph Homophily Measures](https://mlsanity.com/paper/2412.09663/) (2024)
- [Confidence and Stability of Global and Pairwise Scores in NLP Evaluation](https://mlsanity.com/paper/2507.01633/) (2025)
- [Selecting a classification performance measure: matching the measure to the problem](https://mlsanity.com/paper/2409.12391/) (2024)
- [Never mind the metrics -- what about the uncertainty? Visualising confusion matrix metric distributions](https://mlsanity.com/paper/2206.02157/) (2022)
- [FairGridSearch: A Framework to Compare Fairness-Enhancing Models](https://mlsanity.com/paper/2401.02183/) (2024)
- [Vision-Language In-Context Learning Driven Few-Shot Visual Inspection Model](https://mlsanity.com/paper/2502.09057/) (2025)
- [Beyond Surrogates: A Quantitative Analysis for Inter-Metric Relationships](https://mlsanity.com/paper/2603.07671/) (2026)
- [A Unified Algebraic Framework for Classification Performance Evaluation](https://mlsanity.com/paper/2607.04028/) (2026)

## Abstract (arXiv)

Several performance measures can be used for evaluating classification results: accuracy, F-measure, and many others. Can we say that some of them are better than others, or, ideally, choose one measure that is best in all situations? To answer this question, we conduct a systematic analysis of classification performance measures: we formally define a list of desirable properties and theoretically analyze which measures satisfy which properties. We also prove an impossibility theorem: some desirable properties cannot be simultaneously satisfied. Finally, we propose a new family of measures satisfying all desirable properties except one. This family includes the Matthews Correlation Coefficient and a so-called Symmetric Balanced Accuracy that was not previously used in classification literature. We believe that our systematic approach gives an important tool to practitioners for adequately evaluating classification results.