mlsanityData to 14 Oct 2026

Hybrid Adaptive Conformal Offline Reinforcement Learning for Fair Population Health Management

arXiv 2509.09772 ↗First posted 11 Sep 2025cs.LG stat.APcited by 0

On the map: Offline RL and imitation learning from data → Safe offline RL

Abstract (arXiv)

Population health management programs for Medicaid populations coordinate longitudinal outreach and services (e.g., benefits navigation, behavioral health, social needs support, and clinical scheduling) and must be safe, fair, and auditable. We present a Hybrid Adaptive Conformal Offline Reinforcement Learning (HACO) framework that separates risk calibration from preference optimization to generate conservative action recommendations at scale. In our setting, each step involves choosing among common coordination actions (e.g., which member to contact, by which modality, and whether to route to a specialized service) while controlling the near-term risk of adverse utilization events (e.g., unplanned emergency department visits or hospitalizations). Using a de-identified operational dataset from Waymark comprising 2.77 million sequential decisions across 168,126 patients, HACO (i) trains a lightweight risk model for adverse events, (ii) derives a conformal threshold to mask unsafe actions at a target risk level, and (iii) learns a preference policy on the resulting safe subset. We evaluate policies with a version-agnostic fitted Q evaluation (FQE) on stratified subsets and audit subgroup performance across age, sex, and race. HACO achieves strong risk discrimination (AUC ~0.81) with a calibrated threshold ( τ ~0.038 at α = 0.10), while maintaining high safe coverage. Subgroup analyses reveal systematic differences in estimated value across demographics, underscoring the importance of fairness auditing. Our results show that conformal risk gating integrates cleanly with offline RL to deliver conservative, auditable decision support for population health management teams.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safe offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 10 of its 19 references that are in the map. First posted on arXiv 11 Sep 2025.

Builds on

10 of 19 references.

References · 10

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  2. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  3. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  4. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  5. 2019Conformalized Quantile Regressioncited by 629
  6. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 590
  7. 2015Doubly Robust Off-policy Value Evaluation for Reinforcement Learningcited by 339
  8. 2019Batch Policy Learning under Constraintscited by 244
  9. 2019Minimax Weight and Q-Function Learning for Off-Policy Evaluationcited by 150
  10. 2022Calibrated Selective Classificationcited by 33

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Feasibility-Guided Fair Adaptive Offline Reinforcement Learning for Medicaid Care Management
  2. 2025Test-Time Learning and Inference-Time Deliberation for Efficiency-First Offline Reinforcement Learning in Care Coordination and Population Health Management
  3. 2022Semi-Markov Offline Reinforcement Learning for Healthcare
  4. 2021Model Selection for Offline Reinforcement Learning: Practical Considerations for Healthcare Settings
  5. 2024Reinforcement Learning in Dynamic Treatment Regimes Needs Critical Reexamination
  6. 2022Offline Policy Evaluation and Optimization under Confounding
  7. 2026Learning to Compress Time-to-Control: A Reinforcement Learning Framework for Chronic Disease Management
  8. 2022Federated Offline Reinforcement Learning