# HumSet: Dataset of Multilingual Information Extraction and Classification for Humanitarian Crisis Response

arXiv: https://arxiv.org/abs/2210.04573 · HTML: https://mlsanity.com/paper/2210.04573/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 39 references that are in the map.
- First posted on arXiv 10 Oct 2022.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [Text Summarization with Pretrained Encoders](https://mlsanity.com/paper/1908.08345/) (2019)
- [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://mlsanity.com/paper/2012.14740/) (2020)
- [Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting](https://mlsanity.com/paper/1901.09451/) (2019)
- [Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology](https://mlsanity.com/paper/1906.04571/) (2019)
- [Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets](https://mlsanity.com/paper/1908.07898/) (2019)
- [Adversarial Removal of Demographic Attributes from Text Data](https://mlsanity.com/paper/1808.06640/) (2018)
- [MultiVerS: Improving scientific claim verification with weak supervision and full-document context](https://mlsanity.com/paper/2112.01640/) (2021)
- [Neural Document Summarization by Jointly Learning to Score and Select Sentences](https://mlsanity.com/paper/1807.02305/) (2018)
- [Twitter as a Lifeline: Human-annotated Twitter Corpora for NLP of Crisis-related Messages](https://mlsanity.com/paper/1605.05894/) (2016)
- [XtremeDistil: Multi-stage Distillation for Massive Multilingual Models](https://mlsanity.com/paper/2004.05686/) (2020)
- [Societal Biases in Retrieved Contents: Measurement Framework and Adversarial Mitigation for BERT Rankers](https://mlsanity.com/paper/2104.13640/) (2021)
- [Do Neural Ranking Models Intensify Gender Bias?](https://mlsanity.com/paper/2005.00372/) (2020)
- [A scikit-based Python environment for performing multi-label classification](https://mlsanity.com/paper/1702.01460/) (2017)
- [TripClick: The Log Files of a Large Health Web Search Engine](https://mlsanity.com/paper/2103.07901/) (2021)
- [Grep-BiasIR: A Dataset for Investigating Gender Representation-Bias in Information Retrieval Results](https://mlsanity.com/paper/2201.07754/) (2022)
- [Do Perceived Gender Biases in Retrieval Results Affect Relevance Judgements?](https://mlsanity.com/paper/2203.01731/) (2022)

## Built on by (most cited first, 7 of 7)

- [Leave no Place Behind: Improved Geolocation in Humanitarian Documents](https://mlsanity.com/paper/2309.02914/) (2023)
- [Leveraging Domain Knowledge for Inclusive and Bias-aware Humanitarian Response Entry Classification](https://mlsanity.com/paper/2305.16756/) (2023)
- [ScaLearn: Simple and Highly Parameter-Efficient Task Transfer by Learning to Scale](https://mlsanity.com/paper/2310.01217/) (2023)
- [HumVI: A Multilingual Dataset for Detecting Violent Incidents Impacting Humanitarian Aid](https://mlsanity.com/paper/2410.06370/) (2024)
- [Revealing Trends in Datasets from the 2022 ACL and EMNLP Conferences](https://mlsanity.com/paper/2404.08666/) (2024)
- [TextMineX: Data, Evaluation Framework and Ontology-guided LLM Pipeline for Humanitarian Mine Action](https://mlsanity.com/paper/2509.15098/) (2025)
- [QuanReview: Offline, Auditable Reconciliation of Human and LLM Span Annotations](https://mlsanity.com/paper/2609.35685/) (2026)

## Abstract (arXiv)

Timely and effective response to humanitarian crises requires quick and accurate analysis of large amounts of text data - a process that can highly benefit from expert-assisted NLP systems trained on validated and annotated data in the humanitarian response domain. To enable creation of such NLP systems, we introduce and release HumSet, a novel and rich multilingual dataset of humanitarian response documents annotated by experts in the humanitarian response community. The dataset provides documents in three languages (English, French, Spanish) and covers a variety of humanitarian crises from 2018 to 2021 across the globe. For each document, HUMSET provides selected snippets (entries) as well as assigned classes to each entry annotated using common humanitarian information analysis frameworks. HUMSET also provides novel and challenging entry extraction and multi-label entry classification tasks. In this paper, we take a first step towards approaching these tasks and conduct a set of experiments on Pre-trained Language Models (PLM) to establish strong baselines for future research in this domain. The dataset is available at https://blog.thedeep.io/humset/.