# Hitting time for Markov decision process

arXiv: https://arxiv.org/abs/2205.03476 · HTML: https://mlsanity.com/paper/2205.03476/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 2 of its 6 references that are in the map.
- First posted on arXiv 6 May 2022.

## Builds on (most cited first, 2 of 2)

- [Cross-Domain Imitation Learning via Optimal Transport](https://mlsanity.com/paper/2110.03684/) (2021)
- [Hitting Times in Markov Chains with Restart and their Application to Network Centrality](https://mlsanity.com/paper/1503.08548/) (2015)

## Abstract (arXiv)

We define the hitting time for a Markov decision process (MDP). We do not use the hitting time of the Markov process induced by the MDP because the induced chain may not have a stationary distribution. Even it has a stationary distribution, the stationary distribution may not coincide with the (normalized) occupancy measure of the MDP. We observe a relationship between the MDP and the PageRank. Using this observation, we construct an MP whose stationary distribution coincides with the normalized occupancy measure of the MDP and we define the hitting time of the MDP as the hitting time of the associated MP.