mlsanityData to 7 Oct 2026

Why only Micro-F1? Class Weighting of Measures for Relation Classification

arXiv 2205.09460 ↗First posted 19 May 2022cs.CL cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Relation classification models are conventionally evaluated using only a single measure, e.g., micro-F1, macro-F1 or AUC. In this work, we analyze weighting schemes, such as micro and macro, for imbalanced datasets. We introduce a framework for weighting schemes, where existing schemes are extremes, and two new intermediate schemes. We show that reporting results of different weighting schemes better highlights strengths and weaknesses of a model.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 24 of its 48 references that are in the map. First posted on arXiv 19 May 2022.

Builds on

24 of 48 references.

References · 24

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  3. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 987
  4. 2019Matching the Blanks: Distributional Similarity for Relation Learningcited by 271
  5. 2018FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluationcited by 254
  6. 2019DocRED: A Large-Scale Document-Level Relation Extraction Datasetcited by 239
  7. 2021PTR: Prompt Tuning with Rules for Text Classificationcited by 238
  8. 2021How Many Data Points is a Prompt Worth?cited by 197
  9. 2019Dice Loss for Data-imbalanced NLP Taskscited by 180
  10. 2019Macro F1 and Macro F1cited by 70
  11. 2020We Need to Talk About Random Splitscited by 70
  12. 2021FLEX: Unifying Evaluation for Few-Shot NLPcited by 67
12 more
  1. 2019SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominalscited by 63
  2. 2021Changing the World by Changing the Datacited by 51
  3. 2020Demographics Should Not Be the Reason of Toxicity: Mitigating Discrimination in Text Classifications with Instance Weightingcited by 41
  4. 2019Fine-tuning Pre-Trained Transformer Language Models to Distantly Supervised Relation Extractioncited by 36
  5. 2021Bidimensional Leaderboards: Generate and Evaluate Language Hand in Handcited by 32
  6. 2019On Model Stability as a Function of Random Seedcited by 31
  7. 2021Relation Classification with Entity Type Restrictioncited by 31
  8. 2020The Curse of Performance Instability in Analysis Datasets: Consequences, Source, and Suggestionscited by 22
  9. 2021Automatic Construction of Evaluation Suites for Natural Language Generation Datasetscited by 18
  10. 2022Good Classification Measures and How to Find Themcited by 11
  11. 2020NLPStatTest: A Toolkit for Comparing NLP System Performancecited by 3
  12. 2019REflex: Flexible Framework for Relation Extraction in Multiple Domainscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 0; 2024: 4; 2025: 1; 2026: 1.

8 papers.

  1. 2022A Survey of Methods for Addressing Class Imbalance in Deep-Learning Based Natural Language Processingcited by 20
  2. 2022CrossRE: A Cross-Domain Dataset for Relation Extractioncited by 20
  3. 2024ProLLM: Protein Chain-of-Thoughts Enhanced LLM for Protein-Protein Interaction Predictioncited by 12
  4. 2024Multi-Stage Balanced Distillation: Addressing Long-Tail Challenges in Sequence-Level Knowledge Distillationcited by 5
  5. 2025Diagnosing Failures in Large Language Models' Answers: Integrating Error Attribution into Evaluation Frameworkcited by 3
  6. 2024Multi-Branch Mutual-Distillation Transformer for EEG-Based Seizure Subtype Classificationcited by 2
  7. 2024Intertwined Biases Across Social Media Spheres: Unpacking Correlations in Media Bias Dimensionscited by 0
  8. 2026The Word and the Way: Strategies for Domain-Specific BERT Pre-Training in German Medical NLPcited by 0