mlsanityData to 14 Oct 2026

Certifiably Robust Policies for Uncertain Parametric Environments

arXiv 2408.03093 ↗First posted 6 Aug 2024cs.LG cs.AI cs.SYcited by 7

Not on the map. Why?

Abstract (arXiv)

We present a data-driven approach for producing policies that are provably robust across unknown stochastic environments. Existing approaches can learn models of a single environment as an interval Markov decision processes (IMDP) and produce a robust policy with a probably approximately correct (PAC) guarantee on its performance. However these are unable to reason about the impact of environmental parameters underlying the uncertainty. We propose a framework based on parametric Markov decision processes (MDPs) with unknown distributions over parameters. We learn and analyse IMDPs for a set of unknown sample environments induced by parameters. The key challenge is then to produce meaningful performance guarantees that combine the two layers of uncertainty: (1) multiple environments induced by parameters with an unknown distribution; (2) unknown induced environments which are approximated by IMDPs. We present a novel approach based on scenario optimisation that yields a single PAC guarantee quantifying the risk level for which a specified performance level can be assured in unseen environments, plus a means to trade-off risk and performance. We implement and evaluate our framework using multiple robust policy generation methods on a range of benchmarks. We show that our approach produces tight bounds on a policy's performance with high confidence.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 18 of its 58 references that are in the map. First posted on arXiv 6 Aug 2024.

Builds on

18 of 58 references.

References · 18

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  2. 2017Distral: Robust Multitask Reinforcement Learningcited by 233
  3. 2018Meta-Reinforcement Learning of Structured Exploration Strategiescited by 130
  4. 2021Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observabilitycited by 109
  5. 2023A Tutorial on Meta-Reinforcement Learningcited by 108
  6. 2021Online Robust Reinforcement Learning with Model Uncertaintycited by 96
  7. 2019A Bayesian Approach to Robust Reinforcement Learningcited by 34
  8. 2015Safety-Constrained Reinforcement Learning for MDPscited by 33
  9. 2019PAC Statistical Model Checking for Markov Decision Processes and Stochastic Gamescited by 32
  10. 2022Robust Anytime Learning of Markov Decision Processescited by 16
  11. 2014Multiple-Environment Markov Decision Processescited by 13
  12. 2023Train Hard, Fight Easy: Robust Meta Reinforcement Learningcited by 12
6 more
  1. 2015Faster Statistical Model Checking for Unbounded Temporal Propertiescited by 9
  2. 2021Scenario-Based Verification of Uncertain Parametric MDPscited by 8
  3. 2022Sampling-Based Verification of CTMCs with Uncertain Ratescited by 5
  4. 2024What Are the Odds? Improving the foundations of Statistical Model Checkingcited by 5
  5. 2023Learning Robust Policies for Uncertain Parametric Markov Decision Processescited by 4
  6. 2017Multi-Objective Approaches to Markov Decision Processes with Uncertain Transition Parameterscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

  1. 2025Multi-Environment POMDPs: Discrete Model Uncertainty Under Partial Observabilitycited by 3
  2. 2025Data-Efficient Safe Policy Improvement Using Parametric Structurecited by 2
  3. 2026Robust Shielding for Safe Reinforcement Learningcited by 2
  4. 2025Efficient Solution and Learning of Robust Factored MDPscited by 1
  5. 2026Probabilistic Performance Guarantees for Multi-Task Reinforcement Learningcited by 1
  6. 2026Robust Parameter Learning for Uncertain MDPscited by 1
  7. 2026Robust Counterfactuals for Markov Decision Processes under Statistical and Causal Uncertaintiescited by 0