# From Reward Signal to Visual Utility: A Controlled Audit of Medical VLM Post-Training

arXiv: https://arxiv.org/abs/2609.31450 · HTML: https://mlsanity.com/paper/2609.31450/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 13 of its 17 references that are in the map.
- First posted on arXiv 25 Sep 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Medical vision-language models](https://mlsanity.com/mllm/medical-vision-language-models/)

## Builds on (most cited first, 13 of 13)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Sequence Level Training with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06732/) (2015)
- [SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answering](https://mlsanity.com/paper/2102.09542/) (2021)
- [PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answering](https://mlsanity.com/paper/2305.10415/) (2023)
- [Perception-Aware Policy Optimization for Multimodal Reasoning](https://mlsanity.com/paper/2507.06448/) (2025)
- ["My Answer is C": First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models](https://mlsanity.com/paper/2402.14499/) (2024)
- [Ground-R1: Incentivizing Grounded Visual Reasoning via Reinforcement Learning](https://mlsanity.com/paper/2505.20272/) (2025)
- [MedVLThinker: Simple Baselines for Multimodal Medical Reasoning](https://mlsanity.com/paper/2508.02669/) (2025)
- [An Empirical Study on Parameter-Efficient Fine-Tuning for MultiModal Large Language Models](https://mlsanity.com/paper/2406.05130/) (2024)
- [Beyond Accuracy: Evaluating Visual Grounding In Multimodal Medical Reasoning](https://mlsanity.com/paper/2603.03437/) (2026)
- [Do Medical Vision Language Models Actually See? A Counterfactual Grounding Framework and Hard-Negative Contrastive Training for Visually-Reliant Medical VLMs](https://mlsanity.com/paper/2607.03647/) (2026)

## Abstract (arXiv)

Medical vision-language model (VLM) post-training is commonly evaluated through answer accuracy. We examine how changes in accuracy and training objectives relate to image-conditioned decisions in a controlled Qwen2.5-VL-3B study on PMC-VQA. We compare supervised fine-tuning (SFT) with low-rank adaptation (LoRA) restricted to the language model, expanded multimodal adaptation scopes, standard answer-only Group Relative Policy Optimization (GRPO), and a counterfactual evidence objective. On 2,000 clean-test questions, language model LoRA SFT changes correct-image accuracy by +1.10 percentage points (95% paired bootstrap CI:-0.85 to +3.05), while visual-benefit events decrease by 2.40 points and image sensitivity decreases by 5.60 points. Paired records reveal 155 acquired and 203 lost visual-benefit events. Broader adaptation yields lower correct-image accuracy than language-model LoRA SFT. Standard GRPO produces mixed-reward groups and parameter updates, with an uncertain clean test accuracy change. A generation audit reveals that canonical option scores can follow a different token path from generated answers. With scores taken along the greedy generation path, the evidence target improves on the training set; its gains over standard GRPO remain inconsistent on validation data at matched training doses. Sample-level analyses trace how evidence scores, decision margins, and generated answers change during post-training. This empirical and measurement audit identifies gaps between optimization activity, target acquisition, and useful held-out visual behavior.