# Transformer-based Multi-task Learning for Disaster Tweet Categorisation

arXiv: https://arxiv.org/abs/2110.08010 · HTML: https://mlsanity.com/paper/2110.08010/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 37 references that are in the map.
- First posted on arXiv 15 Oct 2021.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [An Empirical Study of Multi-Task Learning on BERT for Biomedical Text Mining](https://mlsanity.com/paper/2005.02799/) (2020)
- [CrisisBERT: a Robust Transformer for Crisis Classification and Contextual Crisis Embedding](https://mlsanity.com/paper/2005.06627/) (2020)
- [Fine-tuning BERT for Joint Entity and Relation Extraction in Chinese Medical Text](https://mlsanity.com/paper/1908.07721/) (2019)
- [On Identifying Hashtags in Disaster Twitter Data](https://mlsanity.com/paper/2001.01323/) (2020)

## Built on by (most cited first, 5 of 5)

- [Harnessing Large Language Models for Disaster Management: A Survey](https://mlsanity.com/paper/2501.06932/) (2025)
- [Graph Neural Network Enhanced Language Models for Efficient Multilingual Text Classification](https://mlsanity.com/paper/2203.02912/) (2022)
- [Enhancing Crisis-Related Tweet Classification with Entity-Masked Language Modeling and Multi-Task Learning](https://mlsanity.com/paper/2211.11468/) (2022)
- [UCD-CS at TREC 2021 Incident Streams Track](https://mlsanity.com/paper/2112.03737/) (2021)
- [Information Type Classification with Contrastive Task-Specialized Sentence Encoders](https://mlsanity.com/paper/2312.11020/) (2023)

## Abstract (arXiv)

Social media has enabled people to circulate information in a timely fashion, thus motivating people to post messages seeking help during crisis situations. These messages can contribute to the situational awareness of emergency responders, who have a need for them to be categorised according to information types (i.e. the type of aid services the messages are requesting). We introduce a transformer-based multi-task learning (MTL) technique for classifying information types and estimating the priority of these messages. We evaluate the effectiveness of our approach with a variety of metrics by submitting runs to the TREC Incident Streams (IS) track: a research initiative specifically designed for disaster tweet classification and prioritisation. The results demonstrate that our approach achieves competitive performance in most metrics as compared to other participating runs. Subsequently, we find that an ensemble approach combining disparate transformer encoders within our approach helps to improve the overall effectiveness to a significant extent, achieving state-of-the-art performance in almost every metric. We make the code publicly available so that our work can be reproduced and used as a baseline for the community for future work in this domain.