# Multi-Task Learning for Visually Grounded Reasoning in Gastrointestinal VQA

arXiv: https://arxiv.org/abs/2511.04384 · HTML: https://mlsanity.com/paper/2511.04384/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 12 references that are in the map.
- First posted on arXiv 6 Nov 2025.

## Builds on (most cited first, 10 of 10)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Kvasir-SEG: A Segmented Polyp Dataset](https://mlsanity.com/paper/1911.07069/) (2019)
- [Image Segmentation Using Text and Image Prompts](https://mlsanity.com/paper/2112.10003/) (2021)
- [Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks](https://mlsanity.com/paper/2311.06242/) (2023)
- [Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models](https://mlsanity.com/paper/2409.17146/) (2024)
- [12-in-1: Multi-Task Vision and Language Representation Learning](https://mlsanity.com/paper/1912.02315/) (2019)
- [Kvasir-Instrument: Diagnostic and therapeutic tool segmentation dataset in gastrointestinal endoscopy](https://mlsanity.com/paper/2011.08065/) (2020)
- [Kvasir-VQA: A Text-Image Pair GI Tract Dataset](https://mlsanity.com/paper/2409.01437/) (2024)
- [Kvasir-VQA-x1: A Multimodal Dataset for Medical Reasoning and Robust MedVQA in Gastrointestinal Endoscopy](https://mlsanity.com/paper/2506.09958/) (2025)
- [Medico 2025: Visual Question Answering for Gastrointestinal Imaging](https://mlsanity.com/paper/2508.10869/) (2025)

## Built on by (most cited first, 2 of 2)

- [Beyond the Leaderboard: Design Lessons for Trustworthy Multimodal VQA](https://mlsanity.com/paper/2607.15241/) (2026)
- [Towards Grounded GI Endoscopy VQA via Multi-Task Learning on Small VLMs](https://mlsanity.com/paper/2607.27122/) (2026)

## Abstract (arXiv)

We present a multi-task framework for the MediaEval Medico 2025 challenge, leveraging a LoRA-tuned Florence-2 model for simultaneous visual question answering (VQA), explanation generation, and visual grounding. The proposed system integrates three curated datasets: (1) Kvasir-VQA-x1 for question-answer learning, (2) a synthetically enriched explanation dataset offering structured medical reasoning, and (3) text-to-region pairs linking visual features with segmentation masks. This multi-task setup enables the model to jointly learn visual grounding, reasoning, and interpretation, producing responses that are both accurate and interpretable. Extensive evaluation demonstrates that our approach substantially improves over single-task baselines in both answer accuracy and visual localization, highlighting the effectiveness of grounded multi-task learning for medical VQA applications.