mlsanityData to 7 Oct 2026

Towards Safe Policy Improvement for Non-Stationary MDPs

arXiv 2010.12645 ↗First posted 23 Oct 2020cs.LG cs.AIcited by 26

Not on the map. Why?

Abstract (arXiv)

Many real-world sequential decision-making problems involve critical systems with financial risks and human-life risks. While several works in the past have proposed methods that are safe for deployment, they assume that the underlying problem is stationary. However, many real-world problems of interest exhibit non-stationarity, and when stakes are high, the cost associated with a false stationarity assumption may be unacceptable. We take the first steps towards ensuring safety, with high confidence, for smoothly-varying non-stationary decision problems. Our proposed method extends a type of safe algorithm, called a Seldonian algorithm, through a synthesis of model-free reinforcement learning with time-series analysis. Safety is ensured using sequential hypothesis testing of a policy's forecasted performance, and confidence intervals are obtained using wild bootstrap.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 5; 2023: 5; 2024: 1; 2025: 2; 2026: 8.

26 papers.

Sort:

By area:

  1. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  2. 2021Universal Off-Policy Evaluationcited by 43
  3. 2022An Adaptive Deep RL Method for Non-Stationary Environments with Piecewise Stable Contextcited by 15
  4. 2021Learning-to-defer for sequential medical decision-making under uncertaintycited by 10
  5. 2023Safe Policy Improvement for POMDPs via Finite-State Controllerscited by 10
  6. 2022Continual Auxiliary Task Learningcited by 9
  7. 2023More for Less: Safe Policy Improvement With Stronger Performance Guaranteescited by 9
  8. 2022No More Pesky Hyperparameters: Offline Hyperparameter Tuning for RLcited by 6
  9. 2023Risk Sensitive Dead-end Identification in Safety-Critical Offline Reinforcement Learningcited by 6
  10. 2026Self-Evolving Agents with Anytime-Valid Certificatescited by 6
  11. 2022Low Variance Off-policy Evaluation with State-based Importance Samplingcited by 5
  12. 2026Conformal Policy Controlcited by 5
14 more
  1. 2023Tempo Adaptation in Non-stationary Reinforcement Learningcited by 3
  2. 2021Coarse-Grained Smoothness for RL in Metric Spacescited by 2
  3. 2022Demystifying Reinforcement Learning in Time-Varying Systemscited by 2
  4. 2023Off-Policy Evaluation for Action-Dependent Non-Stationary Environmentscited by 2
  5. 2024Pausing Policy Learning in Non-stationary Reinforcement Learningcited by 2
  6. 2025Data-Efficient Safe Policy Improvement Using Parametric Structurecited by 2
  7. 2026Robust Probabilistic Shielding for Safe Offline Reinforcement Learningcited by 2
  8. 2025Seldonian Reinforcement Learning for Ad Hoc Teamworkcited by 1
  9. 2026Safe Continual Reinforcement Learning Methods for Nonstationary Environments. Towards a Survey of the State of the Artcited by 1
  10. 2026Value Bonuses using Ensemble Errors for Exploration in Reinforcement Learningcited by 1
  11. 2021Is the Rush to Machine Learning Jeopardizing Safety? Results of a Surveycited by 0
  12. 2026Safe Continual Reinforcement Learning in Non-stationary Environmentscited by 0
  13. 2026Safe Continual Reinforcement Learning under Nonstationarity via Adaptive Safety Constraintscited by 0
  14. 2026When Validation Stops Learning: Auditing Update Admission for Continual Embodied Agentscited by 0