# Macro F1 and Macro F1

arXiv: https://arxiv.org/abs/1911.03347 · HTML: https://mlsanity.com/paper/1911.03347/

## Key facts

- Cited by 70 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Nov 2019.

## Built on by (most cited first, 30 of 70)

- [When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applications](https://mlsanity.com/paper/2310.18339/) (2023)
- [REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformers](https://mlsanity.com/paper/2302.02041/) (2023)
- [Efficient Benchmarking of Language Models](https://mlsanity.com/paper/2308.11696/) (2023)
- [Automated Medical Coding on MIMIC-III and MIMIC-IV: A Critical Review and Replicability Study](https://mlsanity.com/paper/2304.10909/) (2023)
- [xCodeEval: A Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval](https://mlsanity.com/paper/2303.03004/) (2023)
- [A Closer Look at Classification Evaluation Metrics and a Critical Reflection of Common Evaluation Practice](https://mlsanity.com/paper/2404.16958/) (2024)
- [Aegis: Automated Error Generation and Attribution for Multi-Agent Systems](https://mlsanity.com/paper/2509.14295/) (2025)
- [Language Models As Semantic Indexers](https://mlsanity.com/paper/2310.07815/) (2023)
- [RAG-Enhanced Collaborative LLM Agents for Drug Discovery](https://mlsanity.com/paper/2502.17506/) (2025)
- [Multimodal analysis of the predictability of hand-gesture properties](https://mlsanity.com/paper/2108.05762/) (2021)
- [Metrics Matter in Surgical Phase Recognition](https://mlsanity.com/paper/2305.13961/) (2023)
- [Why only Micro-F1? Class Weighting of Measures for Relation Classification](https://mlsanity.com/paper/2205.09460/) (2022)
- [Financial Fine-tuning a Large Time Series Model](https://mlsanity.com/paper/2412.09880/) (2024)
- [Data Leakage and Evaluation Issues in Micro-Expression Analysis](https://mlsanity.com/paper/2211.11425/) (2022)
- [Monitoring crop phenology with street-level imagery using computer vision](https://mlsanity.com/paper/2112.09190/) (2021)
- [SHREC 2022: pothole and crack detection in the road pavement using images and RGB-D data](https://mlsanity.com/paper/2205.13326/) (2022)
- [Guided Semi-Supervised Non-negative Matrix Factorization on Legal Documents](https://mlsanity.com/paper/2201.13324/) (2022)
- [RigoBERTa: A State-of-the-Art Language Model For Spanish](https://mlsanity.com/paper/2205.10233/) (2022)
- [GLADIS: A General and Large Acronym Disambiguation Benchmark](https://mlsanity.com/paper/2302.01860/) (2023)
- [Crop identification using deep learning on LUCAS crop cover photos](https://mlsanity.com/paper/2305.04994/) (2023)
- [Wireless Ad Hoc Federated Learning: A Fully Distributed Cooperative Machine Learning](https://mlsanity.com/paper/2205.11779/) (2022)
- [Epicurus at SemEval-2023 Task 4: Improving Prediction of Human Values behind Arguments by Leveraging Their Definitions](https://mlsanity.com/paper/2302.13925/) (2023)
- [Consistent algorithms for multi-label classification with macro-at-k metrics](https://mlsanity.com/paper/2401.16594/) (2024)
- [Calibrated simplex-mapping classification](https://mlsanity.com/paper/2103.02926/) (2021)
- [Simpson's Bias in NLP Training](https://mlsanity.com/paper/2103.11795/) (2021)
- [Under-bagging Nearest Neighbors for Imbalanced Classification](https://mlsanity.com/paper/2109.00531/) (2021)
- [Automating Wood Species Detection and Classification in Microscopic Images of Fibrous Materials with Deep Learning](https://mlsanity.com/paper/2307.09588/) (2023)
- [Deep Learning and LLM-based Methods Applied to Stellar Lightcurve Classification](https://mlsanity.com/paper/2404.10757/) (2024)
- [Hippocampus: An Efficient and Scalable Memory Module for Agentic AI](https://mlsanity.com/paper/2602.13594/) (2026)
- [Parameter Selection: Why We Should Pay More Attention to It](https://mlsanity.com/paper/2107.05393/) (2021)

## Abstract (arXiv)

The 'macro F1' metric is frequently used to evaluate binary, multi-class and multi-label classification problems. Yet, we find that there exist two different formulas to calculate this quantity. In this note, we show that only under rare circumstances the two computations can be considered equivalent. More specifically, one formula well 'rewards' classifiers which produce a skewed error type distribution. In fact, the difference in outcome of the two computations can be as high as 0.5. The two computations may not only diverge in their scalar result but can also lead to different classifier rankings.