mlsanityData to 7 Oct 2026

Free Form Medical Visual Question Answering in Radiology

arXiv 2401.13081 ↗First posted 23 Jan 2024cs.CV cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Visual Question Answering (VQA) in the medical domain presents a unique, interdisciplinary challenge, combining fields such as Computer Vision, Natural Language Processing, and Knowledge Representation. Despite its importance, research in medical VQA has been scant, only gaining momentum since 2018. Addressing this gap, our research delves into the effective representation of radiology images and the joint learning of multimodal representations, surpassing existing methods. We innovatively augment the SLAKE dataset, enabling our model to respond to a more diverse array of questions, not limited to the immediate content of radiology or pathology images. Our model achieves a top-1 accuracy of 79.55% with a less complex architecture, demonstrating comparable performance to current state-of-the-art models. This research not only advances medical VQA but also opens avenues for practical applications in diagnostic settings.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 31 of its 48 references that are in the map. First posted on arXiv 23 Jan 2024.

Builds on

31 of 48 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2016Densely Connected Convolutional Networkscited by 6,691
  6. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  7. 2019Publicly Available Clinical BERT Embeddingscited by 995
  8. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 969
  9. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
  10. 2022MedCLIP: Contrastive Learning from Unpaired Medical Images and Textcited by 564
  11. 2019ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmissioncited by 487
  12. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 467
19 more
  1. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
  2. 2018Rethinking ImageNet Pre-trainingcited by 376
  3. 2019A large annotated medical image dataset for the development and evaluation of segmentation algorithmscited by 371
  4. 2019Transfusion: Understanding Transfer Learning for Medical Imagingcited by 366
  5. 2021How Much Can CLIP Benefit Vision-and-Language Tasks?cited by 321
  6. 2019PadChest: A large chest x-ray image dataset with multi-label annotated reportscited by 308
  7. 2020CHAOS Challenge -- Combined (CT-MR) Healthy Abdominal Organ Segmentationcited by 307
  8. 2016Hierarchical Question-Image Co-Attention for Visual Question Answeringcited by 278
  9. 2018Bilinear Attention Networkscited by 277
  10. 2021TorchXRayVision: A library of chest X-ray datasets and modelscited by 131
  11. 2020A Survey on Negative Transfercited by 120
  12. 2017Multi-modal Factorized Bilinear Pooling with Co-Attention Learning for Visual Question Answeringcited by 117
  13. 2021Medical Visual Question Answering: A Surveycited by 93
  14. 2021Does CLIP Benefit Visual Question Answering in the Medical Domain as Much as it Does in the General Domain?cited by 85
  15. 2017Beyond Bilinear: Generalized Multimodal Factorized High-order Pooling for Visual Question Answeringcited by 79
  16. 2021Cross-Modal Self-Attention with Multi-Task Pre-Training for Medical Visual Question Answeringcited by 26
  17. 2020A Question-Centric Model for Visual Question Answering in Medical Imagingcited by 23
  18. 2020Hierarchical Deep Multi-modal Network for Medical Visual Question Answeringcited by 11
  19. 2023Efficient Representation Learning for Healthcare with Cross-Architectural Self-Supervisioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Benchmarking GPT-5 for Zero-Shot Multimodal Medical Reasoning in Radiology and Radiation Oncologycited by 5
  2. 2025Unified 3D MRI Representations via Sequence-Invariant Contrastive Learningcited by 1
  3. 2025Test-Time Warmup for Multimodal Large Language Modelscited by 0
  4. 2026How Good LLMs Are at Answering Bangla Medical Visual Questions? Dataset and Benchmarkingcited by 0