# Why only Micro-F1? Class Weighting of Measures for Relation Classification

arXiv: https://arxiv.org/abs/2205.09460 · HTML: https://mlsanity.com/paper/2205.09460/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 48 references that are in the map.
- First posted on arXiv 19 May 2022.

## Builds on (most cited first, 24 of 24)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [Matching the Blanks: Distributional Similarity for Relation Learning](https://mlsanity.com/paper/1906.03158/) (2019)
- [FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation](https://mlsanity.com/paper/1810.10147/) (2018)
- [DocRED: A Large-Scale Document-Level Relation Extraction Dataset](https://mlsanity.com/paper/1906.06127/) (2019)
- [PTR: Prompt Tuning with Rules for Text Classification](https://mlsanity.com/paper/2105.11259/) (2021)
- [How Many Data Points is a Prompt Worth?](https://mlsanity.com/paper/2103.08493/) (2021)
- [Dice Loss for Data-imbalanced NLP Tasks](https://mlsanity.com/paper/1911.02855/) (2019)
- [We Need to Talk About Random Splits](https://mlsanity.com/paper/2005.00636/) (2020)
- [Macro F1 and Macro F1](https://mlsanity.com/paper/1911.03347/) (2019)
- [FLEX: Unifying Evaluation for Few-Shot NLP](https://mlsanity.com/paper/2107.07170/) (2021)
- [SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominals](https://mlsanity.com/paper/1911.10422/) (2019)
- [Changing the World by Changing the Data](https://mlsanity.com/paper/2105.13947/) (2021)
- [Demographics Should Not Be the Reason of Toxicity: Mitigating Discrimination in Text Classifications with Instance Weighting](https://mlsanity.com/paper/2004.14088/) (2020)
- [Fine-tuning Pre-Trained Transformer Language Models to Distantly Supervised Relation Extraction](https://mlsanity.com/paper/1906.08646/) (2019)
- [Bidimensional Leaderboards: Generate and Evaluate Language Hand in Hand](https://mlsanity.com/paper/2112.04139/) (2021)
- [On Model Stability as a Function of Random Seed](https://mlsanity.com/paper/1909.10447/) (2019)
- [Relation Classification with Entity Type Restriction](https://mlsanity.com/paper/2105.08393/) (2021)
- [The Curse of Performance Instability in Analysis Datasets: Consequences, Source, and Suggestions](https://mlsanity.com/paper/2004.13606/) (2020)
- [Automatic Construction of Evaluation Suites for Natural Language Generation Datasets](https://mlsanity.com/paper/2106.09069/) (2021)
- [Good Classification Measures and How to Find Them](https://mlsanity.com/paper/2201.09044/) (2022)
- [NLPStatTest: A Toolkit for Comparing NLP System Performance](https://mlsanity.com/paper/2011.13231/) (2020)
- [REflex: Flexible Framework for Relation Extraction in Multiple Domains](https://mlsanity.com/paper/1906.08318/) (2019)

## Built on by (most cited first, 8 of 8)

- [A Survey of Methods for Addressing Class Imbalance in Deep-Learning Based Natural Language Processing](https://mlsanity.com/paper/2210.04675/) (2022)
- [CrossRE: A Cross-Domain Dataset for Relation Extraction](https://mlsanity.com/paper/2210.09345/) (2022)
- [ProLLM: Protein Chain-of-Thoughts Enhanced LLM for Protein-Protein Interaction Prediction](https://mlsanity.com/paper/2405.06649/) (2024)
- [Multi-Stage Balanced Distillation: Addressing Long-Tail Challenges in Sequence-Level Knowledge Distillation](https://mlsanity.com/paper/2406.13114/) (2024)
- [Diagnosing Failures in Large Language Models' Answers: Integrating Error Attribution into Evaluation Framework](https://mlsanity.com/paper/2507.08459/) (2025)
- [Multi-Branch Mutual-Distillation Transformer for EEG-Based Seizure Subtype Classification](https://mlsanity.com/paper/2412.15224/) (2024)
- [Intertwined Biases Across Social Media Spheres: Unpacking Correlations in Media Bias Dimensions](https://mlsanity.com/paper/2408.15406/) (2024)
- [The Word and the Way: Strategies for Domain-Specific BERT Pre-Training in German Medical NLP](https://mlsanity.com/paper/2606.03250/) (2026)

## Abstract (arXiv)

Relation classification models are conventionally evaluated using only a single measure, e.g., micro-F1, macro-F1 or AUC. In this work, we analyze weighting schemes, such as micro and macro, for imbalanced datasets. We introduce a framework for weighting schemes, where existing schemes are extremes, and two new intermediate schemes. We show that reporting results of different weighting schemes better highlights strengths and weaknesses of a model.