# ProMed: Shapley Information Gain Guided Reinforcement Learning for Proactive Medical LLMs

arXiv: https://arxiv.org/abs/2508.13514 · HTML: https://mlsanity.com/paper/2508.13514/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 56 references that are in the map.
- First posted on arXiv 19 Aug 2025.

## Builds on (most cited first, 24 of 24)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [ORPO: Monolithic Preference Optimization without Reference Model](https://mlsanity.com/paper/2403.07691/) (2024)
- [Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning](https://mlsanity.com/paper/2506.01939/) (2025)
- [ReTool: Reinforcement Learning for Strategic Tool Use in LLMs](https://mlsanity.com/paper/2504.11536/) (2025)
- [HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs](https://mlsanity.com/paper/2412.18925/) (2024)
- [WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning](https://mlsanity.com/paper/2411.02337/) (2024)
- [Reinforcement learning](https://mlsanity.com/paper/2405.10369/) (2024)
- [HuatuoGPT, towards Taming Language Model to Be a Doctor](https://mlsanity.com/paper/2305.15075/) (2023)
- [MediQ: Question-Asking LLMs and a Benchmark for Reliable Interactive Clinical Reasoning](https://mlsanity.com/paper/2406.00922/) (2024)
- [CMB: A Comprehensive Medical Benchmark in Chinese](https://mlsanity.com/paper/2308.08833/) (2023)
- [HealthQ: Unveiling Questioning Capabilities of LLM Chains in Healthcare Conversations](https://mlsanity.com/paper/2409.19487/) (2024)
- [Learning to Ask Informative Questions: Enhancing LLMs with Preference Optimization and Expected Information Gain](https://mlsanity.com/paper/2406.17453/) (2024)
- [TPO: Aligning Large Language Models with Multi-branch & Multi-step Preference Trees](https://mlsanity.com/paper/2410.12854/) (2024)
- [Ask Patients with Patience: Enabling LLMs for Human-Centric Medical Dialogue with Grounded Reasoning](https://mlsanity.com/paper/2502.07143/) (2025)
- [Magical: Medical Lay Language Generation via Semantic Invariance and Layperson-tailored Adaptation](https://mlsanity.com/paper/2508.08730/) (2025)
- [Dialogue is Better Than Monologue: Instructing Medical LLMs via Strategical Conversations](https://mlsanity.com/paper/2501.17860/) (2025)

## Built on by (most cited first, 13 of 13)

- [A Survey of Reinforcement Learning for Large Reasoning Models](https://mlsanity.com/paper/2509.08827/) (2025)
- [ATPO: Adaptive Tree Policy Optimization for Multi-Turn Medical Dialogue](https://mlsanity.com/paper/2603.02216/) (2026)
- [MedKGI: Iterative Differential Diagnosis with Medical Knowledge Graphs and Information-Guided Inquiring](https://mlsanity.com/paper/2512.24181/) (2025)
- [ADEPT: Continual Pretraining via Adaptive Expansion and Dynamic Decoupled Tuning](https://mlsanity.com/paper/2510.10071/) (2025)
- [Proactive Reasoning-with-Retrieval Framework for Medical Multimodal Large Language Models](https://mlsanity.com/paper/2510.18303/) (2025)
- [GraphWalker: Patient Analogy Meets Information Gain for Clinical Reasoning with Large Language Models](https://mlsanity.com/paper/2604.06684/) (2026)
- [The Tell-Tale Norm: ℓ2 Magnitude as a Signal for Reasoning Dynamics in Large Language Models](https://mlsanity.com/paper/2606.06188/) (2026)
- [Beyond MedQA: Towards Real-world Clinical Decision Making in the Era of LLMs](https://mlsanity.com/paper/2510.20001/) (2025)
- [Search for Truth from Reasoning: A Dynamic Representation Editing Framework for Steering LLM Trajectories](https://mlsanity.com/paper/2606.28589/) (2026)
- [GraphMed-LT: Patient-Specific Graph Memory with Latent Clinical Thought Refinement for Multi-Turn Medical Conversations](https://mlsanity.com/paper/2510.03536/) (2025)
- [Beyond Information Seeking: Severity-Aware Question Supervision for Proactive Medical Dialogue](https://mlsanity.com/paper/2608.24521/) (2026)
- [When LLM Meets Tree Search: A Systematic View of Inference as Search in Large Language Models](https://mlsanity.com/paper/2608.30395/) (2026)
- [PCQC: Privileged Counterfactual Question Credit for Multi-Turn Medical Dialogue](https://mlsanity.com/paper/2609.27987/) (2026)

## Abstract (arXiv)

Interactive medical questioning is essential in clinical consultations, where physicians must actively gather necessary patient information. Yet existing medical Large Language Models (LLMs) predominantly follow a reactive paradigm, risking diagnostic errors by answering before seeking sufficient details. To bridge this gap, we propose ProMed, a reinforcement learning framework that transitions LLMs toward a proactive paradigm, enabling them to ask clinically valuable questions before decision-making. Central to ProMed is the Shapley Information Gain (SIG) reward, which quantifies a question's clinical utility as the amount of newly acquired information, while considering its contextual importance via Shapley values. We integrate SIG into a two-stage training pipeline: (1) SIG-Guided Model Initialization uses Monte Carlo Tree Search to construct high-reward interaction trajectories for supervision, and (2) SIG-Augmented Policy Optimization, with a novel SIG-guided Reward Distribution Mechanism that prioritizes informative questions for fine-grained optimization. Experiments on partial-information medical benchmarks show that ProMed significantly outperforms state-of-the-art methods by 6.29% on average and delivers a 54.45% gain over the reactive paradigm, and generalizes robustly to out-of-domain cases. Our codes are available at https://github.com/hxxding/ProMed.