# Cost-Sensitive BERT for Generalisable Sentence Classification with Imbalanced Data

arXiv: https://arxiv.org/abs/2003.11563 · HTML: https://mlsanity.com/paper/2003.11563/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Mar 2020.

## Built on by (most cited first, 14 of 14)

- [State-of-the-art generalisation research in NLP: A taxonomy and review](https://mlsanity.com/paper/2210.03050/) (2022)
- [A Survey of Methods for Addressing Class Imbalance in Deep-Learning Based Natural Language Processing](https://mlsanity.com/paper/2210.04675/) (2022)
- [Conformal Prediction for Class-wise Coverage via Augmented Label Rank Calibration](https://mlsanity.com/paper/2406.06818/) (2024)
- [AutoLAW: Augmented Legal Reasoning through Legal Precedent Prediction](https://mlsanity.com/paper/2106.16034/) (2021)
- [Naturalness Evaluation of Natural Language Generation in Task-oriented Dialogues using BERT](https://mlsanity.com/paper/2109.02938/) (2021)
- [Two-Stage Fine-Tuning: A Novel Strategy for Learning Class-Imbalanced Data](https://mlsanity.com/paper/2207.10858/) (2022)
- [OWAdapt: An adaptive loss function for deep learning using OWA operators](https://mlsanity.com/paper/2305.19443/) (2023)
- [Sentiment Analysis of the COVID-related r/Depression Posts](https://mlsanity.com/paper/2108.06215/) (2021)
- [Perspectives of Non-Expert Users on Cyber Security and Privacy: An Analysis of Online Discussions on Twitter](https://mlsanity.com/paper/2206.02156/) (2022)
- [Improving Cause-of-Death Classification from Verbal Autopsy Reports](https://mlsanity.com/paper/2210.17161/) (2022)
- [Sample Size in Natural Language Processing within Healthcare Research](https://mlsanity.com/paper/2309.02237/) (2023)
- [Majority or Minority: Data Imbalance Learning Method for Named Entity Recognition](https://mlsanity.com/paper/2401.11431/) (2024)
- [Diagnosis extraction from unstructured Dutch echocardiogram reports using span- and document-level characteristic classification](https://mlsanity.com/paper/2408.06930/) (2024)
- [Group & Reweight: A Novel Cost-Sensitive Approach to Mitigating Class Imbalance in Network Traffic Classification](https://mlsanity.com/paper/2409.19214/) (2024)

## Abstract (arXiv)

The automatic identification of propaganda has gained significance in recent years due to technological and social changes in the way news is generated and consumed. That this task can be addressed effectively using BERT, a powerful new architecture which can be fine-tuned for text classification tasks, is not surprising. However, propaganda detection, like other tasks that deal with news documents and other forms of decontextualized social communication (e.g. sentiment analysis), inherently deals with data whose categories are simultaneously imbalanced and dissimilar. We show that BERT, while capable of handling imbalanced classes with no additional data augmentation, does not generalise well when the training and test data are sufficiently dissimilar (as is often the case with news sources, whose topics evolve over time). We show how to address this problem by providing a statistical measure of similarity between datasets and a method of incorporating cost-weighting into BERT when the training and test sets are dissimilar. We test these methods on the Propaganda Techniques Corpus (PTC) and achieve the second-highest score on sentence-level propaganda classification.