Markov Chain Concentration with an Application in Reinforcement Learning

Given X1,· ,XN random variables whose joint distribution is given as μ we will use the Martingale Method to show any Lipshitz Function f over these random variables is subgaussian. The Variance parameter however can have a simple expression under certain conditions. For example under the assumption that the random variables follow a Markov Chain and that the function is Lipschitz under a Weighted Hamming Metric. We shall conclude with certain well known techniques from concentration of suprema of random processes with applications in Reinforcement Learning