# Enhancing Crisis-Related Tweet Classification with Entity-Masked Language Modeling and Multi-Task Learning

arXiv: https://arxiv.org/abs/2211.11468 · HTML: https://mlsanity.com/paper/2211.11468/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 49 references that are in the map.
- First posted on arXiv 21 Nov 2022.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [BERTweet: A pre-trained language model for English Tweets](https://mlsanity.com/paper/2005.10200/) (2020)
- [CrisisMMD: Multimodal Twitter Datasets from Natural Disasters](https://mlsanity.com/paper/1805.00713/) (2018)
- [PERL: Pivot-based Domain Adaptation for Pre-trained Deep Contextualized Embedding Models](https://mlsanity.com/paper/2006.09075/) (2020)
- [Domain Adaptation with Adversarial Training and Graph Embeddings](https://mlsanity.com/paper/1805.05151/) (2018)
- [I-AID: Identifying Actionable Information from Disaster-related Tweets](https://mlsanity.com/paper/2008.13544/) (2020)
- [Transformer-based Multi-task Learning for Disaster Tweet Categorisation](https://mlsanity.com/paper/2110.08010/) (2021)
- [Event-Related Bias Removal for Real-time Disaster Events](https://mlsanity.com/paper/2011.00681/) (2020)
- [Interactive Learning for Identifying Relevant Tweets to Support Real-time Situational Awareness](https://mlsanity.com/paper/1908.02588/) (2019)
- [Low-supervision urgency detection and transfer in short crisis messages](https://mlsanity.com/paper/1907.06745/) (2019)

## Built on by (most cited first, 3 of 3)

- [Detecting Actionable Requests and Offers on Social Media During Crises Using LLMs](https://mlsanity.com/paper/2504.16144/) (2025)
- [Combining Deep Neural Reranking and Unsupervised Extraction for Multi-Query Focused Summarization](https://mlsanity.com/paper/2302.01148/) (2023)
- [Leveraging Large Language Models and Topic Modeling for Toxicity Classification](https://mlsanity.com/paper/2411.17876/) (2024)

## Abstract (arXiv)

Social media has become an important information source for crisis management and provides quick access to ongoing developments and critical information. However, classification models suffer from event-related biases and highly imbalanced label distributions which still poses a challenging task. To address these challenges, we propose a combination of entity-masked language modeling and hierarchical multi-label classification as a multi-task learning problem. We evaluate our method on tweets from the TREC-IS dataset and show an absolute performance gain w.r.t. F1-score of up to 10% for actionable information types. Moreover, we found that entity-masking reduces the effect of overfitting to in-domain events and enables improvements in cross-event generalization.