# Markov Chain Concentration with an Application in Reinforcement Learning

arXiv: https://arxiv.org/abs/2301.02926 · HTML: https://mlsanity.com/paper/2301.02926/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 1 of its 11 references that are in the map.
- First posted on arXiv 7 Jan 2023.

## Builds on (most cited first, 1 of 1)

- [Regret Bounds for Reinforcement Learning via Markov Chain Concentration](https://mlsanity.com/paper/1808.01813/) (2018)

## Abstract (arXiv)

Given X1,· ,XN random variables whose joint distribution is given as μ we will use the Martingale Method to show any Lipshitz Function f over these random variables is subgaussian. The Variance parameter however can have a simple expression under certain conditions. For example under the assumption that the random variables follow a Markov Chain and that the function is Lipschitz under a Weighted Hamming Metric. We shall conclude with certain well known techniques from concentration of suprema of random processes with applications in Reinforcement Learning