Concentration Phenomenon for Random Dynamical Systems: An Operator Theoretic Approach

Via operator theoretic methods, we formalize the concentration phenomenon for a given observable `r' of a discrete time Markov chain with `μπ' as invariant ergodic measure, possibly having support on an unbounded state space. The main contribution of this paper is circumventing tedious probabilistic methods with a study of a composition of the Markov transition operator P followed by a multiplication operator defined by er. It turns out that even if the observable/ reward function is unbounded, but for some for some q>2, |er|q rightarrow 2 propto expbig(μπ(r) +frac2qq-2big) and P is hyperbounded with norm control |P|2 rightarrow q < e^frac12[frac12-frac1q], sharp non-asymptotic concentration bounds follow. Transport-entropy inequality ensures the aforementioned upper bound on multiplication operator for all q>2. The role of reversibility in concentration phenomenon is demystified. These results are particularly useful for the reinforcement learning and controls communities as they allow for concentration inequalities w.r.t standard unbounded obersvables/reward functions where exact knowledge of the system is not available, let alone the reversibility of stationary measure.