mlsanityData to 7 Oct 2026

BAPR: Bayesian amnesic piecewise-robust reinforcement learning for non-stationary continuous control

arXiv 2605.16170 ↗First posted 15 May 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Real-world control systems frequently operate under piecewise stationary conditions, where dynamics remain stable for extended periods before undergoing abrupt regime changes. Standard robust RL methods face a fundamental dilemma: a globally conservative policy wastes performance during stable periods, while a locally adaptive policy risks catastrophic failure when the regime changes undetected. We propose BAPR (Bayesian Amnesic Piecewise-Robust SAC), which unifies Bayesian Online Change Detection (BOCD) with robust ensemble RL. The BAPR operator -- a convex combination of mode-conditional Bellman operators weighted by a frozen belief distribution -- is a γ-contraction. A complementary counterexample, machine-verified in Lean~4, establishes a sharp boundary: when beliefs depend on the Q-function, the contraction factor becomes γ + λDelta (where Delta is the mode reward gap), and contraction fails exactly when γ + λDelta ≥ 1. We derive a component-wise formal error budget for the abstract operator -- every component machine-verified -- bounding post-switch recovery; the budget applies to the abstract mode-mixture operator and inherits to the implemented shared-critic algorithm only through the frozen-parameter design intuition. All results are formally verified with no textttsorry (1,145 lines across 3 Lean~4 files, 22 machine-verified theorems). BOCD drives an adaptive conservatism mechanism: the policy becomes maximally conservative after detected change-points and smoothly relaxes as confidence grows, with detection delay O(log(1/δ)). A context-conditioning module trained via RMDM loss provides mode-aware representations from simulator-provided mode IDs at training time and requires no mode labels at deployment.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 31 references that are in the map. First posted on arXiv 15 May 2026.

Builds on

16 of 31 references.

References · 16

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
  4. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  5. 2020Towards Continual Reinforcement Learning: A Review and Perspectivescited by 268
  6. 2019VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learningcited by 210
  7. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  8. 2013Hidden Parameter Markov Decision Processes: A Semiparametric Regression Approach for Discovering Latent Task Parametrizationscited by 68
  9. 2020A Survey of Reinforcement Learning Algorithms for Dynamically Varying Environmentscited by 62
  10. 2020Risk-Sensitive Reinforcement Learning: Near-Optimal Risk-Sample Tradeoff in Regretcited by 57
  11. 2019Non-Stationary Markov Decision Processes, a Worst-Case Approach using Model-Based Reinforcement Learning, Extended versioncited by 36
  12. 2023Natural Actor-Critic for Robust Reinforcement Learning with Function Approximationcited by 35
4 more
  1. 2021Context-Aware Safe Reinforcement Learning for Non-Stationary Environmentscited by 33
  2. 2021CARL: A Benchmark for Contextual and Adaptive Reinforcement Learningcited by 21
  3. 2025Single Agent Robust Deep Reinforcement Learning for Bus Fleet Controlcited by 3
  4. 2026RE-SAC: Disentangling aleatoric and epistemic risks in bus fleet control: A stable and robust ensemble DRL approachcited by 1

Built on by

Not cited yet.