# Kvasir-VQA: A Text-Image Pair GI Tract Dataset

arXiv: https://arxiv.org/abs/2409.01437 · HTML: https://mlsanity.com/paper/2409.01437/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 57 references that are in the map.
- First posted on arXiv 2 Sep 2024.

## Builds on (most cited first, 15 of 15)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Diffusion Models for Medical Image Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2211.07804/) (2022)
- [2017 Robotic Instrument Segmentation Challenge](https://mlsanity.com/paper/1902.06426/) (2019)
- [Prodigy: An Expeditiously Adaptive Parameter-Free Learner](https://mlsanity.com/paper/2306.06101/) (2023)
- [Medical Visual Question Answering: A Survey](https://mlsanity.com/paper/2111.10056/) (2021)
- [MMBERT: Multimodal BERT Pretraining for Improved Medical VQA](https://mlsanity.com/paper/2104.01394/) (2021)
- [Surgical-VQA: Visual Question Answering in Surgical Scenes using Transformer](https://mlsanity.com/paper/2206.11053/) (2022)
- [Kvasir-Instrument: Diagnostic and therapeutic tool segmentation dataset in gastrointestinal endoscopy](https://mlsanity.com/paper/2011.08065/) (2020)
- [Comparative evaluation of instrument segmentation and tracking methods in minimally invasive surgery](https://mlsanity.com/paper/1805.02475/) (2018)
- [Going Deep in Medical Image Analysis: Concepts, Methods, Challenges and Future Directions](https://mlsanity.com/paper/1902.05655/) (2019)
- [Medical Image Captioning via Generative Pretrained Transformers](https://mlsanity.com/paper/2209.13983/) (2022)
- [PolypConnect: Image inpainting for generating realistic gastrointestinal tract images with polyps](https://mlsanity.com/paper/2205.15413/) (2022)

## Built on by (most cited first, 23 of 23)

- [Baichuan-Omni-1.5 Technical Report](https://mlsanity.com/paper/2501.15368/) (2025)
- [Large Language Model Benchmarks in Medical Tasks](https://mlsanity.com/paper/2410.21348/) (2024)
- [Frontiers in Intelligent Colonoscopy](https://mlsanity.com/paper/2410.17241/) (2024)
- [EndoBench: A Comprehensive Evaluation of Multi-Modal Large Language Models for Endoscopy Analysis](https://mlsanity.com/paper/2505.23601/) (2025)
- [Hallucination-Aware Multimodal Benchmark for Gastrointestinal Image Analysis with Large Vision-Language Models](https://mlsanity.com/paper/2505.07001/) (2025)
- [SurgViVQA: Temporally-Grounded Video Question Answering for Surgical Scene Understanding](https://mlsanity.com/paper/2511.03325/) (2025)
- [Prompt to Polyp: Medical Text-Conditioned Image Synthesis with Diffusion Models](https://mlsanity.com/paper/2505.05573/) (2025)
- [HEDGE: Hallucination Estimation via Dense Geometric Entropy for VQA with Vision-Language Models](https://mlsanity.com/paper/2511.12693/) (2025)
- [EndoCogniAgent: Closed-Loop Agentic Reasoning with Self-Consistency Validation for Endoscopic Diagnosis](https://mlsanity.com/paper/2508.07292/) (2025)
- [Medico 2025: Visual Question Answering for Gastrointestinal Imaging](https://mlsanity.com/paper/2508.10869/) (2025)
- [Med-RewardBench: Benchmarking Reward Models and Judges for Medical Multimodal Large Language Models](https://mlsanity.com/paper/2508.21430/) (2025)
- [Multimodal LLM With Hierarchical Mixture-of-Experts for VQA on 3D Brain MRI](https://mlsanity.com/paper/2509.25889/) (2025)
- [Multi-Task Learning for Visually Grounded Reasoning in Gastrointestinal VQA](https://mlsanity.com/paper/2511.04384/) (2025)
- [Challenging Vision-Language Models with Surgical Data: A New Dataset and Broad Benchmarking Study](https://mlsanity.com/paper/2506.06232/) (2025)
- [Multimodal AI for Gastrointestinal Diagnostics: Tackling VQA in MEDVQA-GI 2025](https://mlsanity.com/paper/2507.14544/) (2025)
- [Colon-X: Advancing Intelligent Colonoscopy toward Clinical Reasoning](https://mlsanity.com/paper/2512.03667/) (2025)
- [SAGE: An Expert-Annotated South Asian GI Endoscopy Dataset for Multimodal Learning and Hallucination Analysis](https://mlsanity.com/paper/2606.22144/) (2026)
- [Querying GI Endoscopy Images: A VQA Approach](https://mlsanity.com/paper/2507.21165/) (2025)
- [LEAML: Label-Efficient Adaptation to Out-of-Distribution Visual Tasks for Multimodal Large Language Models](https://mlsanity.com/paper/2510.03232/) (2025)
- [SurgLQA: Scalable Long-Horizon Surgical Video Question Answering](https://mlsanity.com/paper/2605.17915/) (2026)
- [Parameter-Efficient VLMs for Gastrointestinal Endoscopy: Medical Image Generation and Clinical Visual Question Answering](https://mlsanity.com/paper/2605.24792/) (2026)
- [Beyond the Leaderboard: Design Lessons for Trustworthy Multimodal VQA](https://mlsanity.com/paper/2607.15241/) (2026)
- [Measuring and Improving Complex-Atomic Answer Consistency in Endoscopic VQA](https://mlsanity.com/paper/2607.17834/) (2026)

## Abstract (arXiv)

We introduce Kvasir-VQA, an extended dataset derived from the HyperKvasir and Kvasir-Instrument datasets, augmented with question-and-answer annotations to facilitate advanced machine learning tasks in Gastrointestinal (GI) diagnostics. This dataset comprises 6,500 annotated images spanning various GI tract conditions and surgical instruments, and it supports multiple question types including yes/no, choice, location, and numerical count. The dataset is intended for applications such as image captioning, Visual Question Answering (VQA), text-based generation of synthetic medical images, object detection, and classification. Our experiments demonstrate the dataset's effectiveness in training models for three selected tasks, showcasing significant applications in medical image analysis and diagnostics. We also present evaluation metrics for each task, highlighting the usability and versatility of our dataset. The dataset and supporting artifacts are available at https://datasets.simula.no/kvasir-vqa.