# Do Clinical Models Change Treatment Decisions?

arXiv: https://arxiv.org/abs/2605.28129 · HTML: https://mlsanity.com/paper/2605.28129/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 24 references that are in the map.
- First posted on arXiv 27 May 2026.

## Builds on (most cited first, 8 of 8)

- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Limitations of Large Language Models in Clinical Problem-Solving Arising from Inflexible Reasoning](https://mlsanity.com/paper/2502.04381/) (2025)
- [Forget Forgetting: Continual Learning in a World of Abundant Memory](https://mlsanity.com/paper/2502.07274/) (2025)
- [Correct Reasoning Paths Visit Shared Decision Pivots](https://mlsanity.com/paper/2509.21549/) (2025)
- [Generalist Foundation Models Are Not Clinical Enough for Hospital Operations](https://mlsanity.com/paper/2511.13703/) (2025)

## Abstract (arXiv)

Clinical foundation models are evaluated with factual or exam-style medical QA, but treatment decisions must change when patient context changes. We introduce ClinPivot, an auditable treatment-decision benchmark built from biomedical relations and pivoted patient contexts. ClinPivot asks whether models change treatment choices when new clinical constraints shift the action space. We find that strong medical QA performance does not reliably predict decision-making performance: frontier models and task-adapted Qwen variants often fail to change decisions correctly, and model rankings shift across evaluation regimes. Decision-structured supervision improves pivot-sensitive decision-making and medical QA under matched knowledge budgets, while lightweight replay reduces losses in general assistant ability.