mlsanityData to 14 Oct 2026

Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation

arXiv 2010.16061 ↗First posted 11 Oct 2020cs.LG stat.ME stat.MLcited by 425

Not on the map. Why?

Abstract (arXiv)

Commonly used evaluation measures including Recall, Precision, F-Measure and Rand Accuracy are biased and should not be used without clear understanding of the biases, and corresponding identification of chance or base case levels of the statistic. Using these measures a system that performs worse in the objective sense of Informedness, can appear to perform better under any of these commonly used measures. We discuss several concepts and measures that reflect the probability that prediction is informed versus chance. Informedness and introduce Markedness as a dual measure for the probability that prediction is marked versus chance. Finally we demonstrate elegant connections between the concepts of Informedness, Markedness, Correlation and Significance as well as their intuitive relationships with Recall and Precision, and outline the extension from the dichotomous case to the general multi-class case.

Key facts. Cited by 425 papers in the map (data through 14 Oct 2026). First posted on arXiv 11 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 56; 2022: 63; 2023: 52; 2024: 77; 2025: 106; 2026: 71.

425 papers.

Sort:

By area:

  1. 2021Generalized Out-of-Distribution Detection: A Surveycited by 802
  2. 2021TorchXRayVision: A library of chest X-ray datasets and modelscited by 131
  3. 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
  4. 2023The Segment Anything Model (SAM) for Remote Sensing Applications: From Zero to One Shotcited by 83
  5. 2024FlowAR: Scale-wise Autoregressive Image Generation Meets Flow Matchingcited by 80
  6. 2022Assessing the State of Self-Supervised Human Activity Recognition using Wearablescited by 65
  7. 2022Towards a Guideline for Evaluation Metrics in Medical Image Segmentationcited by 60
  8. 2022Unified Optimal Transport Framework for Universal Domain Adaptationcited by 57
  9. 2021On the Adversarial Robustness of Causal Algorithmic Recoursecited by 56
  10. 2024Reasoning-Enhanced Healthcare Predictions with Knowledge Graph Community Retrievalcited by 45
  11. 2023Analyzing Dataset Annotation Quality Management in the Wildcited by 41
  12. 2022ScanEnts3D: Exploiting Phrase-to-3D-Object Correspondences for Improved Visio-Linguistic Models in 3D Scenescited by 32
38 more
  1. 2025ResearcherBench: Evaluating Deep AI Research Systems on the Frontiers of Scientific Inquirycited by 32
  2. 2021Learning Bounds for Open-Set Learningcited by 27
  3. 2024Evaluating Human-AI Collaboration: A Review and Methodological Frameworkcited by 27
  4. 2021Are we using appropriate segmentation metrics? Identifying correlates of human expert perception for CNN training beyond rolling the DICE coefficientcited by 26
  5. 2024A Closer Look at Classification Evaluation Metrics and a Critical Reflection of Common Evaluation Practicecited by 26
  6. 2023AccelTran: A Sparsity-Aware Accelerator for Dynamic Inference with Transformerscited by 24
  7. 2023Object Detection with Transformers: A Reviewcited by 22
  8. 2024AlignRec: Aligning and Training in Multimodal Recommendationscited by 21
  9. 2021Current Status and Performance Analysis of Table Recognition in Document Images with Deep Neural Networkscited by 18
  10. 2024A Survey on Large Language Models from General Purpose to Medical Applications: Datasets, Methodologies, and Evaluationscited by 16
  11. 2024MFC-Bench: Benchmarking Multimodal Fact-Checking with Large Vision-Language Modelscited by 15
  12. 2021Automated Essay Scoring Using Transformer Modelscited by 14
  13. 2022Cross-Language Binary-Source Code Matching with Intermediate Representationscited by 13
  14. 2022Adaptative Perturbation Patterns: Realistic Adversarial Learning for Robust Intrusion Detectioncited by 13
  15. 2023Transformer-based Vulnerability Detection in Code at EditTime: Zero-shot, Few-shot, or Fine-tuning?cited by 13
  16. 2021An Efficient and Scalable Collection of Fly-inspired Voting Units for Visual Place Recognition in Changing Environmentscited by 12
  17. 2022Extending F1 metric, probabilistic approachcited by 12
  18. 2022Good Classification Measures and How to Find Themcited by 11
  19. 2024Enhancing Supply Chain Visibility with Knowledge Graphs and Large Language Modelscited by 11
  20. 2023The Potential of Visual ChatGPT For Remote Sensingcited by 10
  21. 2024Advanced User Credit Risk Prediction Model using LightGBM, XGBoost and Tabnet with SMOTEENNcited by 10
  22. 2024CT-AGRG: Automated Abnormality-Guided Report Generation from 3D Chest CT Volumescited by 10
  23. 2025MAP: Evaluation and Multi-Agent Enhancement of Large Language Models for Inpatient Pathwayscited by 10
  24. 2021A Comparative Study of Deep Learning Classification Methods on a Small Environmental Microorganism Image Dataset (EMDS-6): from Convolutional Neural Networks to Visual Transformerscited by 9
  25. 2021Generalisation effects of predictive uncertainty estimation in deep learning for digital pathologycited by 9
  26. 2022Succinct Differentiation of Disparate Boosting Ensemble Learning Methods for Prognostication of Polycystic Ovary Syndrome Diagnosiscited by 9
  27. 2023Cross-Domain HAR: Few Shot Transfer Learning for Human Activity Recognitioncited by 9
  28. 2025MotionDreamer: One-to-Many Motion Synthesis with Localized Generative Masked Transformercited by 9
  29. 2022Binary Classification for High Dimensional Data using Supervised Non-Parametric Ensemble Methodcited by 8
  30. 2022Machine Learning for Intrusion Detection in Industrial Control Systems: Applications, Challenges, and Recommendationscited by 8
  31. 2022Evaluating Object (mis)Detection from a Safety and Reliability Perspective: Discussion and Measurescited by 8
  32. 2022MedML: Fusing Medical Knowledge and Machine Learning Models for Early Pediatric COVID-19 Hospitalization and Severity Predictioncited by 8
  33. 2023Clinical Decision Transformer: Intended Treatment Recommendation through Goal Promptingcited by 8
  34. 2025Visual Cues Enhance Predictive Turn-Taking for Two-Party Human Interactioncited by 8
  35. 2021Prediction of lung and colon cancer through analysis of histopathological images by utilizing Pre-trained CNN models with visualization of class activation and saliency mapscited by 7
  36. 2021Snippet Policy Network for Multi-class Varied-length ECG Early Classificationcited by 7
  37. 2021Exploring deep learning methods for recognizing rare diseases and their clinical manifestations from textscited by 7
  38. 2023Fairness Evaluation in Text Classification: Machine Learning Practitioner Perspectives of Individual and Group Fairnesscited by 7

50 most cited of 425 All 425 (JSON)