# Risk Filtering and Risk-Averse Control of Markovian Systems Subject to Model Uncertainty

arXiv: https://arxiv.org/abs/2206.09235 · HTML: https://mlsanity.com/paper/2206.09235/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2022.

## Built on by (most cited first, 2 of 2)

- [Robust Reinforcement Learning with Dynamic Distortion Risk Measures](https://mlsanity.com/paper/2409.10096/) (2024)
- [MDP modeling for multi-stage stochastic programs](https://mlsanity.com/paper/2509.22981/) (2025)

## Abstract (arXiv)

We consider a Markov decision process subject to model uncertainty in a Bayesian framework, where we assume that the state process is observed but its law is unknown to the observer. In addition, while the state process and the controls are observed at time t, the actual cost that may depend on the unknown parameter is not known at time t. The controller optimizes total cost by using a family of special risk measures, that we call risk filters and that are appropriately defined to take into account the model uncertainty of the controlled system. These key features lead to non-standard and non-trivial risk-averse control problems, for which we derive the Bellman principle of optimality. We illustrate the general theory on two practical examples: optimal investment and clinical trials.