# Consistency-preserving Visual Question Answering in Medical Imaging

arXiv: https://arxiv.org/abs/2206.13296 · HTML: https://mlsanity.com/paper/2206.13296/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 28 references that are in the map.
- First posted on arXiv 27 Jun 2022.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [LXMERT: Learning Cross-Modality Encoder Representations from Transformers](https://mlsanity.com/paper/1908.07490/) (2019)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [RUBi: Reducing Unimodal Biases in Visual Question Answering](https://mlsanity.com/paper/1906.10169/) (2019)
- [Cycle-Consistency for Robust Visual Question Answering](https://mlsanity.com/paper/1902.05660/) (2019)
- [GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering](https://mlsanity.com/paper/1902.09506/) (2019)
- [Overcoming Data Limitation in Medical Visual Question Answering](https://mlsanity.com/paper/1909.11867/) (2019)
- [VQA-LOL: Visual Question Answering under the Lens of Logic](https://mlsanity.com/paper/2002.08325/) (2020)
- [Sunny and Dark Outside?! Improving Answer Consistency in VQA through Entailed Question Generation](https://mlsanity.com/paper/1909.04696/) (2019)
- [Cross-Modal Self-Attention with Multi-Task Pre-Training for Medical Visual Question Answering](https://mlsanity.com/paper/2105.00136/) (2021)
- [A Question-Centric Model for Visual Question Answering in Medical Imaging](https://mlsanity.com/paper/2003.08760/) (2020)
- [On Incorporating Semantic Prior Knowledge in Deep Learning Through Embedding-Space Constraints](https://mlsanity.com/paper/1909.13471/) (2019)
- [IQ-VQA: Intelligent Visual Question Answering](https://mlsanity.com/paper/2007.04422/) (2020)
- [Image Classification with Consistent Supporting Evidence](https://mlsanity.com/paper/2111.07048/) (2021)

## Built on by (most cited first, 10 of 10)

- [Baichuan-Omni-1.5 Technical Report](https://mlsanity.com/paper/2501.15368/) (2025)
- [Rad-ReStruct: A Novel VQA Benchmark and Method for Structured Radiology Reporting](https://mlsanity.com/paper/2307.05766/) (2023)
- [Surgical-VQLA++: Adversarial Contrastive Learning for Calibrated Robust Visual Question-Localized Answering in Robotic Surgery](https://mlsanity.com/paper/2408.04958/) (2024)
- [Logical Implications for Visual Question Answering Consistency](https://mlsanity.com/paper/2303.09427/) (2023)
- [Localized Questions in Medical Visual Question Answering](https://mlsanity.com/paper/2307.01067/) (2023)
- [Benchmarking Large Multimodal Models for Ophthalmic Visual Question Answering with OphthalWeChat](https://mlsanity.com/paper/2505.19624/) (2025)
- [Targeted Visual Prompting for Medical Visual Question Answering](https://mlsanity.com/paper/2408.03043/) (2024)
- [Visual Question Answering in Ophthalmology: A Progressive and Practical Perspective](https://mlsanity.com/paper/2410.16662/) (2024)
- [HeteroRAG: A Heterogeneous Retrieval-Augmented Generation Framework for Medical Vision Language Tasks](https://mlsanity.com/paper/2508.12778/) (2025)
- [A reinforcement learning approach for VQA validation: an application to diabetic macular edema grading](https://mlsanity.com/paper/2307.09886/) (2023)

## Abstract (arXiv)

Visual Question Answering (VQA) models take an image and a natural-language question as input and infer the answer to the question. Recently, VQA systems in medical imaging have gained popularity thanks to potential advantages such as patient engagement and second opinions for clinicians. While most research efforts have been focused on improving architectures and overcoming data-related limitations, answer consistency has been overlooked even though it plays a critical role in establishing trustworthy models. In this work, we propose a novel loss function and corresponding training procedure that allows the inclusion of relations between questions into the training process. Specifically, we consider the case where implications between perception and reasoning questions are known a-priori. To show the benefits of our approach, we evaluate it on the clinically relevant task of Diabetic Macular Edema (DME) staging from fundus imaging. Our experiments show that our method outperforms state-of-the-art baselines, not only by improving model consistency, but also in terms of overall model accuracy. Our code and data are available at https://github.com/sergiotasconmorales/consistency_vqa.