mlsanityData to 14 Oct 2026

Efficient Sensitivity Analysis for Parametric Robust Markov Chains

arXiv 2305.01473 ↗First posted 1 May 2023cs.LG cs.LO math.OCcited by 2

Not on the map. Why?

Abstract (arXiv)

We provide a novel method for sensitivity analysis of parametric robust Markov chains. These models incorporate parameters and sets of probability distributions to alleviate the often unrealistic assumption that precise probabilities are available. We measure sensitivity in terms of partial derivatives with respect to the uncertain transition probabilities regarding measures such as the expected reward. As our main contribution, we present an efficient method to compute these partial derivatives. To scale our approach to models with thousands of parameters, we present an extension of this method that selects the subset of k parameters with the highest partial derivative. Our methods are based on linear programming and differentiating these programs around a given value for the parameters. The experiments show the applicability of our approach on models with over a million states and thousands of parameters. Moreover, we embed the results within an iterative learning scheme that profits from having access to a dedicated sensitivity analysis.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 22 of its 59 references that are in the map. First posted on arXiv 1 May 2023.

Builds on

22 of 59 references.

References · 22

  1. 2021Convex optimizationcited by 2,515
  2. 2018Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansioncited by 168
  3. 2020The Probabilistic Model Checker Stormcited by 66
  4. 2019PAC Statistical Model Checking for Markov Decision Processes and Stochastic Gamescited by 32
  5. 2018On the Differentiability of the Solution to Convex Optimization Problemscited by 23
  6. 2022Robust Anytime Learning of Markov Decision Processescited by 16
  7. 2021Verifiable and Compositional Reinforcement Learning Systemscited by 13
  8. 2017Parametric Markov Chains: PCTL Complexity and Fraction-free Gaussian Eliminationcited by 11
  9. 2020The Complexity of Reachability in Parametric Markov Decision Processescited by 10
  10. 2022Parameter Synthesis in Markov Models: A Gentle Surveycited by 10
  11. 2023Decision-Making Under Uncertainty: Beyond Probabilitiescited by 9
  12. 2021Convex Optimization for Parameter Synthesis in MDPscited by 8
10 more
  1. 2021Scenario-Based Verification of Uncertain Parametric MDPscited by 8
  2. 2014Smoothed Model Checking for Uncertain Continuous Time Markov Chainscited by 7
  3. 2021Fast Parametric Model Checking through Model Fragmentationcited by 6
  4. 2021Gradient-Descent for Randomized Controllers under Partial Observabilitycited by 6
  5. 2022Abstraction-Refinement for Hierarchical Probabilistic Modelscited by 6
  6. 2017Reachability in Augmented Interval Markov Chainscited by 4
  7. 2022PAC Statistical Model Checking of Mean Payoff in Discrete- and Continuous-Time MDPcited by 4
  8. 2014Sensitivity analysis for finite Markov chains in discrete timecited by 3
  9. 2017Reachability in Parametric Interval Markov Chains using Constraintscited by 1
  10. 2017Automated Experiment Design for Data-Efficient Verification of Parametric Markov Decision Processescited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Formal Verification of Markov Processes with Learned Parameterscited by 1
  2. 2026Robust Parameter Learning for Uncertain MDPscited by 1