# HumAID: Human-Annotated Disaster Incidents Data from Twitter with Deep Learning Benchmarks

arXiv: https://arxiv.org/abs/2104.03090 · HTML: https://mlsanity.com/paper/2104.03090/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 59 references that are in the map.
- First posted on arXiv 7 Apr 2021.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [CrisisMMD: Multimodal Twitter Datasets from Natural Disasters](https://mlsanity.com/paper/1805.00713/) (2018)
- [Processing Social Media Messages in Mass Emergency: A Survey](https://mlsanity.com/paper/1407.7071/) (2014)
- [Twitter as a Lifeline: Human-annotated Twitter Corpora for NLP of Crisis-related Messages](https://mlsanity.com/paper/1605.05894/) (2016)
- [Analysis of Social Media Data using Multimodal Deep Learning for Disaster Response](https://mlsanity.com/paper/2004.11838/) (2020)
- [Deep Learning Benchmarks and Datasets for Social Media Image Classification for Disaster Response](https://mlsanity.com/paper/2011.08916/) (2020)

## Built on by (most cited first, 30 of 39)

- [MEDIC: A Multi-Task Learning Dataset for Disaster Image Classification](https://mlsanity.com/paper/2108.12828/) (2021)
- [Zero- and Few-Shot Prompting with LLMs: A Comparative Study with Fine-tuned Models for Bangla Sentiment Analysis](https://mlsanity.com/paper/2308.10783/) (2023)
- [Harnessing Large Language Models for Disaster Management: A Survey](https://mlsanity.com/paper/2501.06932/) (2025)
- [Location reference recognition from texts: A survey and comparison](https://mlsanity.com/paper/2207.01683/) (2022)
- [JointMatch: A Unified Approach for Diverse and Collaborative Pseudo-Labeling to Semi-Supervised Text Classification](https://mlsanity.com/paper/2310.14583/) (2023)
- [DeCrisisMB: Debiased Semi-Supervised Learning for Crisis Tweet Classification via Memory Bank](https://mlsanity.com/paper/2310.14577/) (2023)
- [CrisisLTLSum: A Benchmark for Local Crisis Event Timeline Extraction and Summarization](https://mlsanity.com/paper/2210.14190/) (2022)
- [BLP-2023 Task 2: Sentiment Analysis](https://mlsanity.com/paper/2310.16183/) (2023)
- [Evaluating Robustness of LLMs on Crisis-Related Microblogs across Events, Information Types, and Linguistic Features](https://mlsanity.com/paper/2412.10413/) (2024)
- [Evaluation of Geographical Distortions in Language Models](https://mlsanity.com/paper/2404.17401/) (2024)
- [Robust Training of Social Media Image Classification Models for Rapid Disaster Response](https://mlsanity.com/paper/2104.04184/) (2021)
- [NADBenchmarks -- a compilation of Benchmark Datasets for Machine Learning Tasks related to Natural Disasters](https://mlsanity.com/paper/2212.10735/) (2022)
- [Continual Distributed Learning for Crisis Management](https://mlsanity.com/paper/2104.12876/) (2021)
- [Cross-Lingual and Cross-Domain Crisis Classification for Low-Resource Scenarios](https://mlsanity.com/paper/2209.02139/) (2022)
- [PORTRAIT: a hybrid aPproach tO cReate extractive ground-TRuth summAry for dIsaster evenT](https://mlsanity.com/paper/2305.11536/) (2023)
- [IKDSumm: Incorporating Key-phrases into BERT for extractive Disaster Tweet Summarization](https://mlsanity.com/paper/2305.11592/) (2023)
- [Monitoring Critical Infrastructure Facilities During Disasters Using Large Language Models](https://mlsanity.com/paper/2404.14432/) (2024)
- [The challenges of temporal alignment on Twitter during crises](https://mlsanity.com/paper/2104.08535/) (2021)
- [STA: Self-controlled Text Augmentation for Improving Text Classifications](https://mlsanity.com/paper/2302.12784/) (2023)
- [CrisisMatch: Semi-Supervised Few-Shot Learning for Fine-Grained Disaster Tweet Classification](https://mlsanity.com/paper/2310.14627/) (2023)
- [ATSumm: Auxiliary information enhanced approach for abstractive disaster Tweet Summarization with sparse training data](https://mlsanity.com/paper/2405.06541/) (2024)
- [PRISM: LLM-Guided Semantic Clustering for High-Precision Topics](https://mlsanity.com/paper/2604.03180/) (2026)
- [Model-Agnostic Meta Learning for Class Imbalance Adaptation](https://mlsanity.com/paper/2604.18759/) (2026)
- [Tweets Under the Rubble: Detection of Messages Calling for Help in Earthquake Disaster](https://mlsanity.com/paper/2302.13403/) (2023)
- [Information Type Classification with Contrastive Task-Specialized Sentence Encoders](https://mlsanity.com/paper/2312.11020/) (2023)
- [ADSumm: Annotated Ground-truth Summary Datasets for Disaster Tweet Summarization](https://mlsanity.com/paper/2405.06551/) (2024)
- [WildFireCan-MMD: A Multimodal Dataset for Classification of User-Generated Content During Wildfires in Canada](https://mlsanity.com/paper/2504.13231/) (2025)
- [A Dynamic Fusion Model for Consistent Crisis Response](https://mlsanity.com/paper/2509.01053/) (2025)
- [Generalizing to Unseen Disaster Events: A Causal View](https://mlsanity.com/paper/2511.10120/) (2025)
- [Extracting Disaster Impacts and Impact Related Locations in Social Media Posts Using Large Language Models](https://mlsanity.com/paper/2511.21753/) (2025)

## Abstract (arXiv)

Social networks are widely used for information consumption and dissemination, especially during time-critical events such as natural disasters. Despite its significantly large volume, social media content is often too noisy for direct use in any application. Therefore, it is important to filter, categorize, and concisely summarize the available content to facilitate effective consumption and decision-making. To address such issues automatic classification systems have been developed using supervised modeling approaches, thanks to the earlier efforts on creating labeled datasets. However, existing datasets are limited in different aspects (e.g., size, contains duplicates) and less suitable to support more advanced and data-hungry deep learning models. In this paper, we present a new large-scale dataset with ~77K human-labeled tweets, sampled from a pool of ~24 million tweets across 19 disaster events that happened between 2016 and 2019. Moreover, we propose a data collection and sampling pipeline, which is important for social media data sampling for human annotation. We report multiclass classification results using classic and deep learning (fastText and transformer) based models to set the ground for future studies. The dataset and associated resources are publicly available. https://crisisnlp.qcri.org/humaid_dataset.html