# On reachability of Markov chains: A long-run average approach

arXiv: https://arxiv.org/abs/1806.00052 · HTML: https://mlsanity.com/paper/1806.00052/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 May 2018.

## Built on by (most cited first, 1 of 1)

- [Probabilistic Safety Guarantee for Stochastic Control Systems Using Average Reward MDPs](https://mlsanity.com/paper/2511.08419/) (2025)

## Abstract (arXiv)

We consider a Markov control model in discrete time with countable both state space and action space. Using the value function of a suitable long-run average reward problem, we study various reachability/controllability problems. First, we characterize the domain of attraction and escape set of the system, and a generalization called p-domain of attraction, using the aforementioned value function. Next, we solve the problem of maximizing the probability of reaching a set A while avoiding a set B. Finally, we consider a constrained version of the previous problem where we ask for the probability of reaching the set B to be bounded. In the finite case, we use linear programming formulations to solve these problems. Finally, we apply our results to a example of an object that navigates under stochastic influence.