# Majority or Minority: Data Imbalance Learning Method for Named Entity Recognition

arXiv: https://arxiv.org/abs/2401.11431 · HTML: https://mlsanity.com/paper/2401.11431/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 39 references that are in the map.
- First posted on arXiv 21 Jan 2024.

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Neural Architectures for Named Entity Recognition](https://mlsanity.com/paper/1603.01360/) (2016)
- [A systematic study of the class imbalance problem in convolutional neural networks](https://mlsanity.com/paper/1710.05381/) (2017)
- [A survey of loss functions for semantic segmentation](https://mlsanity.com/paper/2006.14822/) (2020)
- [A Unified MRC Framework for Named Entity Recognition](https://mlsanity.com/paper/1910.11476/) (2019)
- [Dice Loss for Data-imbalanced NLP Tasks](https://mlsanity.com/paper/1911.02855/) (2019)
- [Continuous Dice Coefficient: a Method for Evaluating Probabilistic Segmentations](https://mlsanity.com/paper/1906.11031/) (2019)
- [Improving automated multiple sclerosis lesion segmentation with a cascaded 3D convolutional neural network approach](https://mlsanity.com/paper/1702.04869/) (2017)
- [Cost-Sensitive BERT for Generalisable Sentence Classification with Imbalanced Data](https://mlsanity.com/paper/2003.11563/) (2020)
- [Learning to Multitask](https://mlsanity.com/paper/1805.07541/) (2018)
- [FinBERT-MRC: financial named entity recognition using BERT under the machine reading comprehension paradigm](https://mlsanity.com/paper/2205.15485/) (2022)
- [Conversion Prediction Using Multi-task Conditional Attention Networks to Support the Creation of Effective Ad Creative](https://mlsanity.com/paper/1905.07289/) (2019)
- [syrapropa at SemEval-2020 Task 11: BERT-based Models Design For Propagandistic Technique and Span Detection](https://mlsanity.com/paper/2008.10163/) (2020)

## Abstract (arXiv)

Data imbalance presents a significant challenge in various machine learning (ML) tasks, particularly named entity recognition (NER) within natural language processing (NLP). NER exhibits a data imbalance with a long-tail distribution, featuring numerous minority classes (i.e., entity classes) and a single majority class (i.e., O-class). This imbalance leads to misclassifications of the entity classes as the O-class. To tackle this issue, we propose a simple and effective learning method named majority or minority (MoM) learning. MoM learning incorporates the loss computed only for samples whose ground truth is the majority class into the loss of the conventional ML model. Evaluation experiments on four NER datasets (Japanese and English) showed that MoM learning improves prediction performance of the minority classes without sacrificing the performance of the majority class and is more effective than widely known and state-of-the-art methods. We also evaluated MoM learning using frameworks as sequential labeling and machine reading comprehension, which are commonly used in NER. Furthermore, MoM learning has achieved consistent performance improvements regardless of language or framework.