# Polish Medical Visual Question Answering: Vision-Language Models Underutilize Visual Evidence

arXiv: https://arxiv.org/abs/2608.12928 · HTML: https://mlsanity.com/paper/2608.12928/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 17 of its 37 references that are in the map.
- First posted on arXiv 13 Aug 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Medical vision-language models](https://mlsanity.com/mllm/medical-vision-language-models/)

## Builds on (most cited first, 17 of 17)

- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence](https://mlsanity.com/paper/2606.19348/) (2026)
- [SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answering](https://mlsanity.com/paper/2102.09542/) (2021)
- [Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering](https://mlsanity.com/paper/1712.00377/) (2017)
- [RUBi: Reducing Unimodal Biases in Visual Question Answering](https://mlsanity.com/paper/1906.10169/) (2019)
- [Overcoming Language Priors in Visual Question Answering with Adversarial Regularization](https://mlsanity.com/paper/1810.03649/) (2018)
- [Artifacts or Abduction: How Do LLMs Answer Multiple-Choice Questions Without the Question?](https://mlsanity.com/paper/2402.12483/) (2024)
- [Which of These Best Describes Multiple Choice Evaluation with LLMs? A) Forced B) Flawed C) Fixable D) All of the Above](https://mlsanity.com/paper/2502.14127/) (2025)
- [Reducing Language Biases in Visual Question Answering with Visually-Grounded Question Encoder](https://mlsanity.com/paper/2007.06198/) (2020)
- [Towards Developing a Multilingual and Code-Mixed Visual Question Answering System by Knowledge Distillation](https://mlsanity.com/paper/2109.04653/) (2021)
- [GPT-4 passes most of the 297 written Polish Board Certification Examinations](https://mlsanity.com/paper/2405.01589/) (2024)
- [Polish-English medical knowledge transfer: A new benchmark and results](https://mlsanity.com/paper/2412.00559/) (2024)
- [LLMzSzL: a comprehensive LLM benchmark for Polish](https://mlsanity.com/paper/2501.02266/) (2025)
- [Annotation-Efficient Vision-Language Model Adaptation to the Polish Language Using the LLaVA Framework](https://mlsanity.com/paper/2602.14073/) (2026)
- [Does Language Shift Break Medical Vision-Language Models? Indonesian Radiology Visual Question Answering Case Study](https://mlsanity.com/paper/2606.03693/) (2026)
- [Reassessing High-Performing LLMs on Polish Medical Exams: True Competence or Bias-Driven Performance?](https://mlsanity.com/paper/2606.12250/) (2026)
- [Jako Tako or Fluent? Presenting PoVisLE: A Polish Vision-Language Evaluation](https://mlsanity.com/paper/2608.07763/) (2026)

## Abstract (arXiv)

We introduce a Polish-language medical visual question answering (VQA) benchmark, built from Polish Board Certification Examination questions for licensed physicians and dentists pursuing specialist certification. The benchmark comprises image-containing questions spanning diverse medical specialties and visual domains, together with a text-only question answering (QA) control set. We evaluate Polish-oriented, general-purpose open-weight, and commercial vision-language models. The task remains challenging: the best model achieves 79.0% accuracy on the full VQA set, and only GPT-5.6 surpasses the approximate human reference on the subset with available candidate responses; all other evaluated models perform worse than humans. To assess visual grounding, we compare complete inputs with configurations omitting the image, the question, or both, and categorize questions by image importance. Models derive more useful information from the question text than from the image and perform worse on image-dominant questions. Across both QA and VQA, they nevertheless achieve above-chance accuracy from the answer choices alone, showing that non-trivial performance can persist even when key task components are missing.