# Parameter-Efficient VLMs for Gastrointestinal Endoscopy: Medical Image Generation and Clinical Visual Question Answering

arXiv: https://arxiv.org/abs/2605.24792 · HTML: https://mlsanity.com/paper/2605.24792/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 28 references that are in the map.
- First posted on arXiv 24 May 2026.

## Builds on (most cited first, 12 of 12)

- [SurgVLM: A Large Vision-Language Model and Systematic Evaluation Benchmark for Surgical Intelligence](https://mlsanity.com/paper/2506.02555/) (2025)
- [Towards a Multimodal Large Language Model with Pixel-Level Insight for Biomedicine](https://mlsanity.com/paper/2412.09278/) (2024)
- [Kvasir-VQA: A Text-Image Pair GI Tract Dataset](https://mlsanity.com/paper/2409.01437/) (2024)
- [EndoBench: A Comprehensive Evaluation of Multi-Modal Large Language Models for Endoscopy Analysis](https://mlsanity.com/paper/2505.23601/) (2025)
- [Hallucination-Aware Multimodal Benchmark for Gastrointestinal Image Analysis with Large Vision-Language Models](https://mlsanity.com/paper/2505.07001/) (2025)
- [Visual Question Answering in the Medical Domain](https://mlsanity.com/paper/2309.11080/) (2023)
- [Guiding Medical Vision-Language Models with Explicit Visual Prompts: Framework Design and Comprehensive Exploration of Prompt Variations](https://mlsanity.com/paper/2501.02385/) (2025)
- [Prompt to Polyp: Medical Text-Conditioned Image Synthesis with Diffusion Models](https://mlsanity.com/paper/2505.05573/) (2025)
- [Biomed-DPT: Dual Modality Prompt Tuning for Biomedical Vision-Language Models](https://mlsanity.com/paper/2505.05189/) (2025)
- [Advancing AI-Powered Medical Image Synthesis: Insights from MedVQA-GI Challenge Using CLIP, Fine-Tuned Stable Diffusion, and Dream-Booth + LoRA](https://mlsanity.com/paper/2502.20667/) (2025)
- [Point, Detect, Count: Multi-Task Medical Image Understanding with Instruction-Tuned Vision-Language Models](https://mlsanity.com/paper/2505.16647/) (2025)
- [Synthetic Data-Driven Multi-Architecture Framework for Automated Polyp Segmentation Through Integrated Detection and Mask Generation](https://mlsanity.com/paper/2508.06170/) (2025)

## Abstract (arXiv)

The major limitations of gastrointestinal (GI) endoscopy AI systems arise from a shortage of annotated data, strict privacy policies, and significant bottlenecks in conventional model fine-tuning. Such limitations impede the successful application of sophisticated AI models in clinical practice, particularly affecting the reliability and scalability of diagnosis. In this paper, we present a dual-pipeline PEFT model that addresses two fundamental problems: medical Visual Question Answering (VQA) and the generation of privacy-preserving synthetic data. For clinical VQA, we adopt the Florence-2 vision-language model. Leveraging PEFT enhances model interpretability while substantially reducing the computational cost of training. Simultaneously, we employ Low-Rank Adaptation (LoRA) with Stable Diffusion 2.1 to generate high-quality GI images that enhance training databases without violating patient privacy. This research utilized the Kvasir-VQA dataset. Our Florence-2 VQA model achieved ROUGE-1 of 0.92, ROUGE-L of 0.91, and BLEU score improvements from 0.08 to 0.24. Fine-tuning on private datasets consistently showed better results than fine-tuning on public datasets. The rank-4 LoRA synthesis achieved optimal performance with a fidelity score of 0.290, an agreement score of 0.730, and a Frechet BiomedCLIP Distance (FBD) of 1450, reducing computational costs by almost 90 percent. This framework improves the clinical potential of AI in GI endoscopy. Compared to FLUX, MSDM, and Kandinsky 2.2, our model demonstrates superior FBD and strong semantic alignment. While other models lead in Fidelity or Agreement, our lower FBD indicates better image-text coherence. These results establish our approach as a robust solution for enhancing VQA and synthetic data generation in clinical AI.