Patient-Centered Treatment Planning for Chronic Multimorbidity: A Hierarchical Reinforcement Learning Framework for Preference Modeling
On the map: Deep reinforcement learning methods → RL for clinical treatment and surgery
Abstract (arXiv)
Patient preference, defined as a patient's demonstrated willingness and capacity to adhere to clinical recommendations, is a primary determinant of therapeutic effect yet remains structurally absent from existing computational treatment planning models. We address this gap by presenting patient-centered factored-action hierarchical option-critic (FAHOC), a hierarchical reinforcement learning (HRL) framework that jointly learns high-level options corresponding to therapeutic strategies and factored intra-option policies that decompose the joint action space into disease- and intervention-specific subcomponents, while imposing a cooperation-aware action masking mechanism. This enables structured exploration, improved credit assignment across hierarchy levels, and more interpretable decision pathways, while enforcing patients' preferences. Formal guarantees establish that cooperative patients achieve higher optimal expected health outcomes than non-cooperative patients, and that the factored Q-function approximation error is provably bounded. The framework is evaluated using longitudinal data collected from approximately 50,000 comorbid hypertension and type 2 diabetes mellitus patients from five hospitals in the Southeast U.S. FAHOC achieves a quality-adjusted life year expectancy equivalent improvement of 0.669 (vs -0.133 observed clinician practice), correctly identifies cooperative patients in 95.9% of cases and never violates a patient's preference in held-out test, demonstrating that HRL with explicit preference constraints can support preference-consistent, clinically safe decision-making in multimorbidity management.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for clinical treatment and surgery, one of 15 directions in Deep reinforcement learning methods. Builds on 10 of its 57 references that are in the map. First posted on arXiv 30 Sep 2026.
Builds on
10 of 57 references.
References · 10
- 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
- 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
- 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
- 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
- 2015Prioritized Experience Replaycited by 1,099
- 2016The Option-Critic Architecturecited by 448
- 2015Doubly Robust Off-policy Value Evaluation for Reinforcement Learningcited by 339
- 2019Batch Policy Learning under Constraintscited by 244
- 2023Leveraging Factored Action Spaces for Efficient Offline Reinforcement Learning in Healthcarecited by 42
- 2024Advancing Multi-Organ Disease Care: A Hierarchical Multi-Agent Reinforcement Learning Frameworkcited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Learning to Compress Time-to-Control: A Reinforcement Learning Framework for Chronic Disease Management
- 2026Boosted Distributional Reinforcement Learning: Analysis and Healthcare Applications
- 2026Learning from Disagreement: Clinician Overrides as Implicit Preference Signals for Clinical AI in Value-Based Care
- 2026The Evidence Ladder for Reinforcement Learning in Healthcare: From Retrospective Policies to Trusted Interventions
- 2024Pruning the Path to Optimal Care: Identifying Systematically Suboptimal Medical Decision-Making with Inverse Reinforcement Learning
- 2025medDreamer: Model-Based Reinforcement Learning with Latent Imagination on Complex EHRs for Clinical Decision Support
- 2025Combining LLM decision and RL action selection to improve RL policy for adaptive interventions
- 2025Reinforcement Learning on Dyads to Enhance Medication Adherence