# Efficient Sensitivity Analysis for Parametric Robust Markov Chains

arXiv: https://arxiv.org/abs/2305.01473 · HTML: https://mlsanity.com/paper/2305.01473/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 59 references that are in the map.
- First posted on arXiv 1 May 2023.

## Builds on (most cited first, 22 of 22)

- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansion](https://mlsanity.com/paper/1807.01675/) (2018)
- [The Probabilistic Model Checker Storm](https://mlsanity.com/paper/2002.07080/) (2020)
- [PAC Statistical Model Checking for Markov Decision Processes and Stochastic Games](https://mlsanity.com/paper/1905.04403/) (2019)
- [On the Differentiability of the Solution to Convex Optimization Problems](https://mlsanity.com/paper/1804.05098/) (2018)
- [Robust Anytime Learning of Markov Decision Processes](https://mlsanity.com/paper/2205.15827/) (2022)
- [Verifiable and Compositional Reinforcement Learning Systems](https://mlsanity.com/paper/2106.05864/) (2021)
- [Parametric Markov Chains: PCTL Complexity and Fraction-free Gaussian Elimination](https://mlsanity.com/paper/1709.02093/) (2017)
- [The Complexity of Reachability in Parametric Markov Decision Processes](https://mlsanity.com/paper/2009.13128/) (2020)
- [Parameter Synthesis in Markov Models: A Gentle Survey](https://mlsanity.com/paper/2207.06801/) (2022)
- [Decision-Making Under Uncertainty: Beyond Probabilities](https://mlsanity.com/paper/2303.05848/) (2023)
- [Convex Optimization for Parameter Synthesis in MDPs](https://mlsanity.com/paper/2107.00108/) (2021)
- [Scenario-Based Verification of Uncertain Parametric MDPs](https://mlsanity.com/paper/2112.13020/) (2021)
- [Smoothed Model Checking for Uncertain Continuous Time Markov Chains](https://mlsanity.com/paper/1402.1450/) (2014)
- [Fast Parametric Model Checking through Model Fragmentation](https://mlsanity.com/paper/2102.01490/) (2021)
- [Gradient-Descent for Randomized Controllers under Partial Observability](https://mlsanity.com/paper/2111.04407/) (2021)
- [Abstraction-Refinement for Hierarchical Probabilistic Models](https://mlsanity.com/paper/2206.02653/) (2022)
- [Reachability in Augmented Interval Markov Chains](https://mlsanity.com/paper/1701.02996/) (2017)
- [PAC Statistical Model Checking of Mean Payoff in Discrete- and Continuous-Time MDP](https://mlsanity.com/paper/2206.01465/) (2022)
- [Sensitivity analysis for finite Markov chains in discrete time](https://mlsanity.com/paper/1408.2029/) (2014)
- [Reachability in Parametric Interval Markov Chains using Constraints](https://mlsanity.com/paper/1706.00270/) (2017)
- [Automated Experiment Design for Data-Efficient Verification of Parametric Markov Decision Processes](https://mlsanity.com/paper/1707.01322/) (2017)

## Built on by (most cited first, 2 of 2)

- [Formal Verification of Markov Processes with Learned Parameters](https://mlsanity.com/paper/2501.15767/) (2025)
- [Robust Parameter Learning for Uncertain MDPs](https://mlsanity.com/paper/2605.01339/) (2026)

## Abstract (arXiv)

We provide a novel method for sensitivity analysis of parametric robust Markov chains. These models incorporate parameters and sets of probability distributions to alleviate the often unrealistic assumption that precise probabilities are available. We measure sensitivity in terms of partial derivatives with respect to the uncertain transition probabilities regarding measures such as the expected reward. As our main contribution, we present an efficient method to compute these partial derivatives. To scale our approach to models with thousands of parameters, we present an extension of this method that selects the subset of k parameters with the highest partial derivative. Our methods are based on linear programming and differentiating these programs around a given value for the parameters. The experiments show the applicability of our approach on models with over a million states and thousands of parameters. Moreover, we embed the results within an iterative learning scheme that profits from having access to a dedicated sensitivity analysis.