# Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation

arXiv: https://arxiv.org/abs/2010.16061 · HTML: https://mlsanity.com/paper/2010.16061/

## Key facts

- Cited by 425 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Oct 2020.

## Built on by (most cited first, 30 of 425)

- [Generalized Out-of-Distribution Detection: A Survey](https://mlsanity.com/paper/2110.11334/) (2021)
- [TorchXRayVision: A library of chest X-ray datasets and models](https://mlsanity.com/paper/2111.00595/) (2021)
- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [The Segment Anything Model (SAM) for Remote Sensing Applications: From Zero to One Shot](https://mlsanity.com/paper/2306.16623/) (2023)
- [FlowAR: Scale-wise Autoregressive Image Generation Meets Flow Matching](https://mlsanity.com/paper/2412.15205/) (2024)
- [Assessing the State of Self-Supervised Human Activity Recognition using Wearables](https://mlsanity.com/paper/2202.12938/) (2022)
- [Towards a Guideline for Evaluation Metrics in Medical Image Segmentation](https://mlsanity.com/paper/2202.05273/) (2022)
- [Unified Optimal Transport Framework for Universal Domain Adaptation](https://mlsanity.com/paper/2210.17067/) (2022)
- [On the Adversarial Robustness of Causal Algorithmic Recourse](https://mlsanity.com/paper/2112.11313/) (2021)
- [Reasoning-Enhanced Healthcare Predictions with Knowledge Graph Community Retrieval](https://mlsanity.com/paper/2410.04585/) (2024)
- [Analyzing Dataset Annotation Quality Management in the Wild](https://mlsanity.com/paper/2307.08153/) (2023)
- [ScanEnts3D: Exploiting Phrase-to-3D-Object Correspondences for Improved Visio-Linguistic Models in 3D Scenes](https://mlsanity.com/paper/2212.06250/) (2022)
- [ResearcherBench: Evaluating Deep AI Research Systems on the Frontiers of Scientific Inquiry](https://mlsanity.com/paper/2507.16280/) (2025)
- [Learning Bounds for Open-Set Learning](https://mlsanity.com/paper/2106.15792/) (2021)
- [Evaluating Human-AI Collaboration: A Review and Methodological Framework](https://mlsanity.com/paper/2407.19098/) (2024)
- [Are we using appropriate segmentation metrics? Identifying correlates of human expert perception for CNN training beyond rolling the DICE coefficient](https://mlsanity.com/paper/2103.06205/) (2021)
- [A Closer Look at Classification Evaluation Metrics and a Critical Reflection of Common Evaluation Practice](https://mlsanity.com/paper/2404.16958/) (2024)
- [AccelTran: A Sparsity-Aware Accelerator for Dynamic Inference with Transformers](https://mlsanity.com/paper/2302.14705/) (2023)
- [Object Detection with Transformers: A Review](https://mlsanity.com/paper/2306.04670/) (2023)
- [AlignRec: Aligning and Training in Multimodal Recommendations](https://mlsanity.com/paper/2403.12384/) (2024)
- [Current Status and Performance Analysis of Table Recognition in Document Images with Deep Neural Networks](https://mlsanity.com/paper/2104.14272/) (2021)
- [A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluations](https://mlsanity.com/paper/2406.10303/) (2024)
- [MFC-Bench: Benchmarking Multimodal Fact-Checking with Large Vision-Language Models](https://mlsanity.com/paper/2406.11288/) (2024)
- [Automated Essay Scoring Using Transformer Models](https://mlsanity.com/paper/2110.06874/) (2021)
- [Cross-Language Binary-Source Code Matching with Intermediate Representations](https://mlsanity.com/paper/2201.07420/) (2022)
- [Adaptative Perturbation Patterns: Realistic Adversarial Learning for Robust Intrusion Detection](https://mlsanity.com/paper/2203.04234/) (2022)
- [Transformer-based Vulnerability Detection in Code at EditTime: Zero-shot, Few-shot, or Fine-tuning?](https://mlsanity.com/paper/2306.01754/) (2023)
- [An Efficient and Scalable Collection of Fly-inspired Voting Units for Visual Place Recognition in Changing Environments](https://mlsanity.com/paper/2109.10986/) (2021)
- [Extending F1 metric, probabilistic approach](https://mlsanity.com/paper/2210.11997/) (2022)
- [Good Classification Measures and How to Find Them](https://mlsanity.com/paper/2201.09044/) (2022)

## Abstract (arXiv)

Commonly used evaluation measures including Recall, Precision, F-Measure and Rand Accuracy are biased and should not be used without clear understanding of the biases, and corresponding identification of chance or base case levels of the statistic. Using these measures a system that performs worse in the objective sense of Informedness, can appear to perform better under any of these commonly used measures. We discuss several concepts and measures that reflect the probability that prediction is informed versus chance. Informedness and introduce Markedness as a dual measure for the probability that prediction is marked versus chance. Finally we demonstrate elegant connections between the concepts of Informedness, Markedness, Correlation and Significance as well as their intuitive relationships with Recall and Precision, and outline the extension from the dichotomous case to the general multi-class case.