mlsanityData to 7 Oct 2026

Macro F1 and Macro F1

arXiv 1911.03347 ↗First posted 8 Nov 2019cs.LG stat.MLcited by 70

Not on the map. Why?

Abstract (arXiv)

The 'macro F1' metric is frequently used to evaluate binary, multi-class and multi-label classification problems. Yet, we find that there exist two different formulas to calculate this quantity. In this note, we show that only under rare circumstances the two computations can be considered equivalent. More specifically, one formula well 'rewards' classifiers which produce a skewed error type distribution. In fact, the difference in outcome of the two computations can be as high as 0.5. The two computations may not only diverge in their scalar result but can also lead to different classifier rankings.

Key facts. Cited by 70 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 14; 2023: 12; 2024: 12; 2025: 14; 2026: 5.

70 papers.

Sort:

By area:

  1. 2023When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applicationscited by 110
  2. 2023REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformerscited by 93
  3. 2023Efficient Benchmarking of Language Modelscited by 71
  4. 2023Automated Medical Coding on MIMIC-III and MIMIC-IV: A Critical Review and Replicability Studycited by 48
  5. 2023xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrievalcited by 46
  6. 2024A Closer Look at Classification Evaluation Metrics and a Critical Reflection of Common Evaluation Practicecited by 26
  7. 2023Language Models As Semantic Indexerscited by 20
  8. 2025Aegis: Automated Error Generation and Attribution for Multi-Agent Systemscited by 19
  9. 2025RAG-Enhanced Collaborative LLM Agents for Drug Discoverycited by 18
  10. 2021Multimodal analysis of the predictability of hand-gesture propertiescited by 14
  11. 2023Metrics Matter in Surgical Phase Recognitioncited by 13
  12. 2022Why only Micro-F1? Class Weighting of Measures for Relation Classificationcited by 8
38 more
  1. 2024Financial Fine-tuning a Large Time Series Modelcited by 8
  2. 2022Data Leakage and Evaluation Issues in Micro-Expression Analysiscited by 7
  3. 2021Monitoring crop phenology with street-level imagery using computer visioncited by 6
  4. 2022SHREC 2022: pothole and crack detection in the road pavement using images and RGB-D datacited by 6
  5. 2022RigoBERTa: A State-of-the-Art Language Model For Spanishcited by 5
  6. 2023GLADIS: A General and Large Acronym Disambiguation Benchmarkcited by 5
  7. 2023Crop identification using deep learning on LUCAS crop cover photoscited by 5
  8. 2022Guided Semi-Supervised Non-negative Matrix Factorization on Legal Documentscited by 4
  9. 2022Wireless Ad Hoc Federated Learning: A Fully Distributed Cooperative Machine Learningcited by 4
  10. 2023Epicurus at SemEval-2023 Task 4: Improving Prediction of Human Values behind Arguments by Leveraging Their Definitionscited by 4
  11. 2024Consistent algorithms for multi-label classification with macro-at-k metricscited by 4
  12. 2021Calibrated simplex-mapping classificationcited by 3
  13. 2021Simpson's Bias in NLP Trainingcited by 3
  14. 2021Under-bagging Nearest Neighbors for Imbalanced Classificationcited by 3
  15. 2023Automating Wood Species Detection and Classification in Microscopic Images of Fibrous Materials with Deep Learningcited by 3
  16. 2024Deep Learning and LLM-based Methods Applied to Stellar Lightcurve Classificationcited by 3
  17. 2026Hippocampus: An Efficient and Scalable Memory Module for Agentic AIcited by 3
  18. 2021Parameter Selection: Why We Should Pay More Attention to Itcited by 2
  19. 2021Communication-Computation Efficient Device-Edge Co-Inference via AutoMLcited by 2
  20. 2021Deep Learning Based Automated COVID-19 Classification from Computed Tomography Imagescited by 2
  21. 2022VISTANet: VIsual Spoken Textual Additive Net for Interpretable Multimodal Emotion Recognitioncited by 2
  22. 2022Interpretable Multimodal Emotion Recognition using Hybrid Fusion of Speech and Image Datacited by 2
  23. 2022Dynamic loss balancing and sequential enhancement for road-safety assessment and traffic scene classificationcited by 2
  24. 2023COVID-19 Detection Using Slices Processing Techniques and a Modified Xception Classifier from Computed Tomography Imagescited by 2
  25. 2024A Study on Self-Supervised Pretraining for Vision Problems in Gastrointestinal Endoscopycited by 2
  26. 2025Evidence for Phenotype-Driven Disparities in Freezing of Gait Detection and Approaches to Bias Mitigationcited by 2
  27. 2025LLM-Match: An Open-Sourced Patient Matching Model Based on Large Language Models and Retrieval-Augmented Generationcited by 2
  28. 2025Meta-PerSER: Few-Shot Listener Personalized Speech Emotion Recognition via Meta-learningcited by 2
  29. 2025Joint Modeling of Entities and Discourse Relations for Coherence Assessmentcited by 2
  30. 2021Improving type information inferred by decompilers with supervised machine learningcited by 1
  31. 2021Custom Deep Neural Network for 3D Covid Chest CT-scan Classificationcited by 1
  32. 2022Power System Anomaly Detection and Classification Utilizing WLS-EKF State Estimation and Machine Learningcited by 1
  33. 2022COVID-19 Detection Using Segmentation, Region Extraction and Classification Pipelinecited by 1
  34. 2024Mining Temporal Attack Patterns from Cyberthreat Intelligence Reportscited by 1
  35. 2024An In-Depth Analysis of Data Reduction Methods for Sustainable Deep Learningcited by 1
  36. 2024Evidence-Enhanced Triplet Generation Framework for Hallucination Alleviation in Generative Question Answeringcited by 1
  37. 2024Longitudinal Ensemble Integration for sequential classification with multimodal datacited by 1
  38. 2025An uncertainty-aware Bayesian framework for machine learning classification models: A case study in land cover classificationcited by 1

50 most cited of 70 All 70 (JSON)