mlsanityData to 14 Oct 2026

Behavioral Cloning Outperforms Entropy-Regularized RL: Critic-Driven Failure of Actor-Critic Methods on Adaptive Tumor Treatment

arXiv 2609.06667 ↗First posted 6 Sep 2026cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Imitation learning methods

also in 1 more
Abstract (arXiv)

Adaptive dosing requires policies that reduce tumor burden without excessive toxicity. Learned dosing policies are typically judged against historical or heuristic comparators, which cannot show whether a policy has found the best behavior available. We instead study a three-population tumor-control ODE in which optimal-control analysis fixes the form of a good schedule -- bang-bang dosing punctuated by a singular arc -- and construct a numerical controller of that form as a proxy for near-optimal behavior. Judged against this reference under a sustained-cure criterion -- 200 consecutive days below 5% carrying capacity -- Soft Actor-Critic (SAC) trained from scratch never reaches cure. Behavioral cloning (BC) of the reference reproduces it (100% sustained cure, 30/30 seeds), but SAC fine-tuning of the cloned policy destroys it across five entropy coefficients, and TD3 and BC-regularized SAC fail identically; the pattern persists under multiplicative pharmacokinetic action noise. Along curative trajectories the post-collapse critic ranks the collapsed-policy action above the reference action in 96% of states, concentrated in the maintenance phase, and the policy settles into a non-curative adaptive-therapy equilibrium. The reference is what makes this legible: against a heuristic comparator the fine-tuned policy would read as a competent controller rather than a failure.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 7 of its 18 references that are in the map. First posted on arXiv 6 Sep 2026.

Builds on

7 of 18 references.

References · 7

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  3. 2017Deep Reinforcement Learning that Matterscited by 754
  4. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  5. 2023Efficient Online Reinforcement Learning with Offline Datacited by 363
  6. 2023Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuningcited by 237
  7. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 176

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Recurrent Deep Reinforcement Learning for Chemotherapy Control under Partial Observability
  2. 2026Adaptive Chemotherapy Control under Tumor Heterogeneity via Reinforcement Learning
  3. 2024Reinforcement Learning for Control of Non-Markovian Cellular Population Dynamics
  4. 2026Belief-Space Control for Personalized Cancer Treatment via Active Inference
  5. 2025CLARITY: Medical World Model for Guiding Treatment Decisions by Simulating Context-Aware Disease Trajectories
  6. 2026Admissible Diffusion for Multimodal Interventional Trajectories
  7. 2025Medical World Model: Generative Simulation of Tumor Evolution for Treatment Planning
  8. 2022Tight Performance Guarantees of Imitator Policies with Continuous Actions