# Question-Conditioned Counterfactual Image Generation for VQA

arXiv: https://arxiv.org/abs/1911.06352 · HTML: https://mlsanity.com/paper/1911.06352/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Nov 2019.

## Built on by (most cited first, 5 of 5)

- [Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directions](https://mlsanity.com/paper/2107.13782/) (2021)
- [ECINN: Efficient Counterfactuals from Invertible Neural Networks](https://mlsanity.com/paper/2103.13701/) (2021)
- [Counterfactual Samples Synthesizing and Training for Robust Visual Question Answering](https://mlsanity.com/paper/2110.01013/) (2021)
- [A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Future](https://mlsanity.com/paper/2412.14056/) (2024)
- [COIN: Counterfactual Image Generation for VQA Interpretation](https://mlsanity.com/paper/2201.03342/) (2022)

## Abstract (arXiv)

While Visual Question Answering (VQA) models continue to push the state-of-the-art forward, they largely remain black-boxes - failing to provide insight into how or why an answer is generated. In this ongoing work, we propose addressing this shortcoming by learning to generate counterfactual images for a VQA model - i.e. given a question-image pair, we wish to generate a new image such that i) the VQA model outputs a different answer, ii) the new image is minimally different from the original, and iii) the new image is realistic. Our hope is that providing such counterfactual examples allows users to investigate and understand the VQA model's internal mechanisms.