# Continual VQA for Disaster Response Systems

arXiv: https://arxiv.org/abs/2209.10320 · HTML: https://mlsanity.com/paper/2209.10320/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 10 references that are in the map.
- First posted on arXiv 21 Sep 2022.

## Builds on (most cited first, 6 of 6)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [FloodNet: A High Resolution Aerial Imagery Dataset for Post Flood Scene Understanding](https://mlsanity.com/paper/2012.02951/) (2020)
- [VQA-Aid: Visual Question Answering for Post-Disaster Damage Assessment and Analysis](https://mlsanity.com/paper/2106.10548/) (2021)
- [An Efficient Modern Baseline for FloodNet VQA](https://mlsanity.com/paper/2205.15025/) (2022)

## Built on by (most cited first, 2 of 2)

- [Task Arithmetic with LoRA for Continual Learning](https://mlsanity.com/paper/2311.02428/) (2023)
- [Unleashing the Potential of Large Language Model: Zero-shot VQA for Flood Disaster Scenario](https://mlsanity.com/paper/2312.01882/) (2023)

## Abstract (arXiv)

Visual Question Answering (VQA) is a multi-modal task that involves answering questions from an input image, semantically understanding the contents of the image and answering it in natural language. Using VQA for disaster management is an important line of research due to the scope of problems that are answered by the VQA system. However, the main challenge is the delay caused by the generation of labels in the assessment of the affected areas. To tackle this, we deployed pre-trained CLIP model, which is trained on visual-image pairs. however, we empirically see that the model has poor zero-shot performance. Thus, we instead use pre-trained embeddings of text and image from this model for our supervised training and surpass previous state-of-the-art results on the FloodNet dataset. We expand this to a continual setting, which is a more real-life scenario. We tackle the problem of catastrophic forgetting using various experience replay methods. Our training runs are available at: https://wandb.ai/compyle/continual_vqa_final. Our code is available at https://github.com/AdityaKane2001/continual_vqa.