mlsanityData to 14 Oct 2026

Overlapping Word Removal is All You Need: Revisiting Data Imbalance in Hope Speech Detection

arXiv 2204.05488 ↗First posted 12 Apr 2022cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Hope Speech Detection, a task of recognizing positive expressions, has made significant strides recently. However, much of the current works focus on model development without considering the issue of inherent imbalance in the data. Our work revisits this issue in hope-speech detection by introducing focal loss, data augmentation, and pre-processing strategies. Accordingly, we find that introducing focal loss as part of Multilingual-BERT's (M-BERT) training process mitigates the effect of class imbalance and improves overall F1-Macro by 0.11. At the same time, contextual and back-translation-based word augmentation with M-BERT improves results by 0.10 over baseline despite imbalance. Finally, we show that overlapping word removal based on pre-processing, though simple, improves F1-Macro by 0.28. In due process, we present detailed studies depicting various behaviors of each of these strategies and summarize key findings from our empirical results for those interested in getting the most out of M-BERT for hope speech detection under real-world conditions of data imbalance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 58 references that are in the map. First posted on arXiv 12 Apr 2022.

Builds on

23 of 58 references.

References · 23

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  5. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  7. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  8. 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 807
  9. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
  10. 2019How multilingual is Multilingual BERT?cited by 645
  11. 2018Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relationscited by 201
  12. 2021MuRIL: Multilingual Representations for Indian Languagescited by 179
11 more
  1. 2017Data Augmentation for Low-Resource Neural Machine Translationcited by 139
  2. 2020M2m: Imbalanced Classification via Major-to-minor Translationcited by 134
  3. 2019Augmenting Data with Mixup for Sentence Classification: An Empirical Studycited by 125
  4. 2018MaskGAN: Better Text Generation via Filling in the______cited by 89
  5. 2018Imbalanced Deep Learning by Minority Class Incremental Rectificationcited by 65
  6. 2020Audio ALBERT: A Lite BERT for Self-supervised Learning of Audio Representationcited by 59
  7. 2017Dual Supervised Learningcited by 33
  8. 2015sense2vec - A Fast and Accurate Method for Word Sense Disambiguation In Neural Word Embeddingscited by 20
  9. 2017On the Role of Text Preprocessing in Neural Network Architectures: An Evaluation Study on Text Categorization and Sentiment Analysiscited by 15
  10. 2020SemGloVe: Semantic Co-occurrences for GloVe from BERTcited by 6
  11. 2021IIITT@LT-EDI-EACL2021-Hope Speech Detection: There is always Hope in Transformerscited by 6

Built on by

Not cited yet.