# CrisisViT: A Robust Vision Transformer for Crisis Image Classification

arXiv: https://arxiv.org/abs/2401.02838 · HTML: https://mlsanity.com/paper/2401.02838/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 43 references that are in the map.
- First posted on arXiv 5 Jan 2024.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language](https://mlsanity.com/paper/2202.03555/) (2022)
- [Scaling Autoregressive Video Models](https://mlsanity.com/paper/1906.02634/) (2019)
- [Dynamic Fusion with Intra- and Inter- Modality Attention Flow for Visual Question Answering](https://mlsanity.com/paper/1812.05252/) (2018)
- [CrisisMMD: Multimodal Twitter Datasets from Natural Disasters](https://mlsanity.com/paper/1805.00713/) (2018)
- [Processing Social Media Messages in Mass Emergency: A Survey](https://mlsanity.com/paper/1407.7071/) (2014)
- [ConvNets vs. Transformers: Whose Visual Representations are More Transferable?](https://mlsanity.com/paper/2108.05305/) (2021)
- [Natural Disasters Detection in Social Media and Satellite imagery: a survey](https://mlsanity.com/paper/1901.04277/) (2019)
- [Incidents1M: a large-scale dataset of images with natural disasters, damage, and incidents](https://mlsanity.com/paper/2201.04236/) (2022)
- [Rapid Damage Assessment Using Social Media Images by Combining Human and Machine Intelligence](https://mlsanity.com/paper/2004.06675/) (2020)
- [Applications of Online Deep Learning for Crisis Response Using Social Media Information](https://mlsanity.com/paper/1610.01030/) (2016)
- [On Identifying Disaster-Related Tweets: Matching-based or Learning-based?](https://mlsanity.com/paper/1705.02009/) (2017)

## Built on by (most cited first, 5 of 5)

- [CFIR: Fast and Effective Long-Text To Image Retrieval for Large Corpora](https://mlsanity.com/paper/2402.15276/) (2024)
- [QuakeSet: A Dataset and Low-Resource Models to Monitor Earthquakes through Sentinel-1](https://mlsanity.com/paper/2403.18116/) (2024)
- [CLCE: An Approach to Refining Cross-Entropy and Contrastive Learning for Optimized Learning Fusion](https://mlsanity.com/paper/2402.14551/) (2024)
- [Multimodal Learning with Augmentation Techniques for Natural Disaster Assessment](https://mlsanity.com/paper/2511.00004/) (2025)
- [Explaining the Unseen: Multimodal Vision-Language Reasoning for Situational Awareness in Underground Mining Disasters](https://mlsanity.com/paper/2512.09092/) (2025)

## Abstract (arXiv)

In times of emergency, crisis response agencies need to quickly and accurately assess the situation on the ground in order to deploy relevant services and resources. However, authorities often have to make decisions based on limited information, as data on affected regions can be scarce until local response services can provide first-hand reports. Fortunately, the widespread availability of smartphones with high-quality cameras has made citizen journalism through social media a valuable source of information for crisis responders. However, analyzing the large volume of images posted by citizens requires more time and effort than is typically available. To address this issue, this paper proposes the use of state-of-the-art deep neural models for automatic image classification/tagging, specifically by adapting transformer-based architectures for crisis image classification (CrisisViT). We leverage the new Incidents1M crisis image dataset to develop a range of new transformer-based image classification models. Through experimentation over the standard Crisis image benchmark dataset, we demonstrate that the CrisisViT models significantly outperform previous approaches in emergency type, image relevance, humanitarian category, and damage severity classification. Additionally, we show that the new Incidents1M dataset can further augment the CrisisViT models resulting in an additional 1.25% absolute accuracy gain.