# World Feedback for Clinical Agents: Diagnosing RL in FHIR Environments

arXiv: https://arxiv.org/abs/2607.01470 · HTML: https://mlsanity.com/paper/2607.01470/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 14 references that are in the map.
- First posted on arXiv 1 Jul 2026.

## Builds on (most cited first, 9 of 9)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [Scaling Laws for Reward Model Overoptimization](https://mlsanity.com/paper/2210.10760/) (2022)
- [FHIR-AgentBench: Benchmarking LLM Agents for Realistic Interoperable EHR Question Answering](https://mlsanity.com/paper/2509.19319/) (2025)
- [Imbalanced Gradients in RL Post-Training of Multi-Task LLMs](https://mlsanity.com/paper/2510.19178/) (2025)
- [Multi-Task GRPO: Reliable LLM Reasoning Across Tasks](https://mlsanity.com/paper/2602.05547/) (2026)
- [HealthAdminBench: Evaluating Computer-Use Agents on Healthcare Administration Tasks](https://mlsanity.com/paper/2604.09937/) (2026)

## Abstract (arXiv)

Clinical protocol-execution tasks -- checking a lab value, applying a threshold, placing a correctly structured FHIR order -- are natural candidates for RL from world feedback: once clinical SMEs encode decision logic into a verifier, that verifier grades unlimited rollouts without per-episode annotation. But applying RL requires a sound feedback channel and sufficient base capability. We audit MedAgentBench v1/v2, find a 41.7% silent-finish ceiling that makes inaction the RL dominant strategy, and construct MedAgentBench-v3 (MAB-v3) (508 tasks, 8.9% ceiling). Training Qwen3-8B exposes two structural barriers: a capability ceiling (10/20 task types have 0% base performance, zero gradient) and a format-knowledge barrier (3/20 types require exact clinical codes undiscoverable by exploration). Pure RL reaches 18.2% pass@1 vs. 34.1% for rule-based SFT; the 15.9~pp gap is attributable entirely to these barriers. A decision/format-knowledge/lookup taxonomy predicts RL learnability and prescribes the fix: SFT to inject codes, RL to learn conditionals.