mlsanityData to 7 Oct 2026

Multimodal Categorization of Crisis Events in Social Media

arXiv 2004.04917 ↗First posted 10 Apr 2020cs.LG cs.AI cs.CLcited by 34

Not on the map. Why?

Abstract (arXiv)

Recent developments in image classification and natural language processing, coupled with the rapid growth in social media usage, have enabled fundamental advances in detecting breaking events around the world in real-time. Emergency response is one such area that stands to gain from these advances. By processing billions of texts and images a minute, events can be automatically detected to enable emergency response workers to better assess rapidly evolving situations and deploy resources accordingly. To date, most event detection techniques in this area have focused on image-only or text-only approaches, limiting detection performance and impacting the quality of information delivered to crisis response teams. In this paper, we present a new multimodal fusion method that leverages both images and texts as input. In particular, we introduce a cross-attention module that can filter uninformative and misleading components from weak modalities on a sample by sample basis. In addition, we employ a multimodal graph-based approach to stochastically transition between embeddings of different multimodal pairs during training to better regularize the learning process as well as dealing with limited training data by constructing new matched pairs from different samples. We show that our method outperforms the unimodal approaches and strong multimodal baselines by a large margin on three crisis-related tasks.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 5; 2023: 4; 2024: 7; 2025: 11; 2026: 3.

34 papers.

Sort:

By area:

  1. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Surveycited by 74
  2. 2021Multimodal Knowledge Expansioncited by 26
  3. 2022Incidents1M: a large-scale dataset of images with natural disasters, damage, and incidentscited by 25
  4. 2025Harnessing Large Language Models for Disaster Management: A Surveycited by 20
  5. 2025Aligning Vision to Language: Annotation-Free Multimodal Knowledge Graph Construction for Enhanced LLMs Reasoningcited by 19
  6. 2021Multimodal Fake News Detectioncited by 14
  7. 2022Adaptive Contrastive Learning on Multimodal Transformer for Review Helpfulness Predictionscited by 13
  8. 2022SANCL: Multimodal Review Helpfulness Prediction with Selective Attention and Natural Contrastive Learningcited by 12
  9. 2024A social context-aware graph-based multimodal attentive learning framework for disaster content classification during emergencies: a benchmark dataset and methodcited by 12
  10. 2023Gradient-Boosted Decision Tree for Listwise Context Model in Multimodal Review Helpfulness Predictioncited by 7
  11. 2023Understanding Social Media Cross-Modality Discourse in Linguistic Spacecited by 6
  12. 2023Overcome Modal Bias in Multi-modal Federated Learning via Balanced Modality Selectioncited by 6
22 more
  1. 2024CrisisKAN: Knowledge-infused and Explainable Multimodal Attention Network for Crisis Event Classificationcited by 6
  2. 2021Robust Training of Social Media Image Classification Models for Rapid Disaster Responsecited by 4
  3. 2024Resolving Sentiment Discrepancy for Multimodal Sentiment Detection via Semantics Completion and Decompositioncited by 4
  4. 2022VIDI: A Video Dataset of Incidentscited by 3
  5. 2024Harnessing Shared Relations via Multimodal Mixup Contrastive Learning for Multimodal Classificationcited by 2
  6. 2021Multimodal Breast Lesion Classification Using Cross-Attention Deep Networkscited by 1
  7. 2022GaLeNet: Multimodal Learning for Disaster Prediction, Management and Reliefcited by 1
  8. 2023Balanced Multi-modal Federated Learning via Cross-Modal Infiltrationcited by 1
  9. 2024More is Less? A Simulation-Based Approach to Dynamic Interactions between Biases in Multimodal Modelscited by 1
  10. 2025Efficient Traffic Forecasting on Large-Scale Road Network by Regularized Adaptive Graph Convolutioncited by 1
  11. 2025CAMO: Causality-Guided Adversarial Multimodal Domain Generalization for Crisis Classificationcited by 1
  12. 2024Interactive Event Sifting using Bayesian Graph Neural Networkscited by 0
  13. 2025WildFireCan-MMD: A Multimodal Dataset for Classification of User-Generated Content During Wildfires in Canadacited by 0
  14. 2025PREMISE: Matching-based Prediction for Accurate Review Recommendationcited by 0
  15. 2025Differential Attention for Multimodal Crisis Event Analysiscited by 0
  16. 2025Multimodal Fine-grained Reasoning for Post Quality Evaluationcited by 0
  17. 2025Contextual Attention-Based Multimodal Fusion of LLM and CNN for Sentiment Analysiscited by 0
  18. 2025Cross-Attention Multimodal Fusion for Breast Cancer Diagnosis: Integrating Mammography and Clinical Data with Explainabilitycited by 0
  19. 2025Multimodal Learning with Augmentation Techniques for Natural Disaster Assessmentcited by 0
  20. 2026Cross-Modal Rationale Transfer for Explainable Humanitarian Classification on Social Mediacited by 0
  21. 2026A Multimodal Deep Learning Framework for Edema Classification Using HCT and Clinical Datacited by 0
  22. 2026Towards reliable multimodal disaster severity assessment through preference optimization and explainable vision-language reasoningcited by 0