# Treatment, evidence, imitation, and chat

arXiv: https://arxiv.org/abs/2506.23040 · HTML: https://mlsanity.com/paper/2506.23040/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 111 references that are in the map.
- First posted on arXiv 29 Jun 2025.

## Builds on (most cited first, 26 of 26)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [Behavioral Cloning from Observation](https://mlsanity.com/paper/1805.01954/) (2018)
- [Reinforcement Learning in Healthcare: A Survey](https://mlsanity.com/paper/1908.08796/) (2019)
- [Deep Learning: A Critical Appraisal](https://mlsanity.com/paper/1801.00631/) (2018)
- [MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning](https://mlsanity.com/paper/2406.00922/) (2024)
- [Deep Reinforcement Learning for Sepsis Treatment](https://mlsanity.com/paper/1711.09602/) (2017)
- [Continuous State-Space Models for Optimal Sepsis Treatment - a Deep Reinforcement Learning Approach](https://mlsanity.com/paper/1705.08422/) (2017)
- [Estimating Dynamic Treatment Regimes in Mobile Health Using V-learning](https://mlsanity.com/paper/1611.03531/) (2016)
- [Evaluating Reinforcement Learning Algorithms in Observational Health Settings](https://mlsanity.com/paper/1805.12298/) (2018)
- [Interpretable Off-Policy Evaluation in Reinforcement Learning by Highlighting Influential Transitions](https://mlsanity.com/paper/2002.03478/) (2020)
- [POPCORN: Partially Observed Prediction COnstrained ReiNforcement Learning](https://mlsanity.com/paper/2001.04032/) (2020)
- [Superhuman performance of a large language model on the reasoning tasks of a physician](https://mlsanity.com/paper/2412.10849/) (2024)
- [PEGASUS: A Policy Search Method for Large MDPs and POMDPs](https://mlsanity.com/paper/1301.3878/) (2013)
- [DiReCT: Diagnostic Reasoning for Clinical Notes via Large Language Models](https://mlsanity.com/paper/2408.01933/) (2024)
- [Efficient Evaluation of Natural Stochastic Policies in Offline Reinforcement Learning](https://mlsanity.com/paper/2006.03886/) (2020)
- [Can Large Language Models Match the Conclusions of Systematic Reviews?](https://mlsanity.com/paper/2505.22787/) (2025)
- [MedPAIR: Measuring Physicians and AI Relevance Alignment in Medical Question Answering](https://mlsanity.com/paper/2505.24040/) (2025)
- [Medical Knowledge Integration into Reinforcement Learning Algorithms for Dynamic Treatment Regimes](https://mlsanity.com/paper/2407.00364/) (2024)
- [Relative Sparsity for Medical Decision Problems](https://mlsanity.com/paper/2211.16566/) (2022)
- [Adaptive Experiment Design with Synthetic Controls](https://mlsanity.com/paper/2401.17205/) (2024)
- [Large Language Models as Co-Pilots for Causal Inference in Medical Studies](https://mlsanity.com/paper/2407.19118/) (2024)

## Abstract (arXiv)

Large language models are thought to have the potential to aid in medical decision making. This work investigates the degree to which this might be the case. We start with the treatment problem, the patient's core medical decision-making task, which is solved in collaboration with a clinician. We discuss different approaches to solving it, including, within evidence-based medicine, experimental and observational data. We then discuss the chat problem, and how this differs from the treatment problem -- in particular with respect to imitation (and how imitation alone cannot solve the true treatment problem, although this does not mean it is not useful). We then discuss how a large-language-model-based system might be trained to solve the treatment problem, highlighting that the major challenges relate to the ethics of experimentation and the assumptions associated with observation. We finally discuss how these challenges relate to evidence-based medicine and how this might inform the efforts of the medical research community to solve the treatment problem. Throughout, we illustrate our arguments with the cholesterol medications, statins.