# Concentration Phenomenon for Random Dynamical Systems: An Operator Theoretic Approach

arXiv: https://arxiv.org/abs/2212.03670 · HTML: https://mlsanity.com/paper/2212.03670/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 29 references that are in the map.
- First posted on arXiv 7 Dec 2022.

## Builds on (most cited first, 10 of 10)

- [Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator](https://mlsanity.com/paper/1801.05039/) (2018)
- [Learning Without Mixing: Towards A Sharp Analysis of Linear System Identification](https://mlsanity.com/paper/1802.08334/) (2018)
- [Learning nonlinear dynamical systems from a single trajectory](https://mlsanity.com/paper/2004.14681/) (2020)
- [Non-asymptotic and Accurate Learning of Nonlinear Dynamical Systems](https://mlsanity.com/paper/2002.08538/) (2020)
- [Learning with little mixing](https://mlsanity.com/paper/2206.08269/) (2022)
- [Least-Squares Temporal Difference Learning for the Linear Quadratic Regulator](https://mlsanity.com/paper/1712.08642/) (2017)
- [Stochastic Gradient Descent Learns State Equations with Nonlinear Activations](https://mlsanity.com/paper/1809.03019/) (2018)
- [Adaptive Approximate Policy Iteration](https://mlsanity.com/paper/2002.03069/) (2020)
- [Unknown mixing times in apprenticeship and reinforcement learning](https://mlsanity.com/paper/1905.09704/) (2019)
- [Transportation-Inequalities, Lyapunov Stability and Sampling for Dynamical Systems on Continuous State Space](https://mlsanity.com/paper/2205.12448/) (2022)

## Built on by (most cited first, 1 of 1)

- [Learning and Concentration for High Dimensional Linear Gaussians: an Invariant Subspace Approach](https://mlsanity.com/paper/2304.01708/) (2023)

## Abstract (arXiv)

Via operator theoretic methods, we formalize the concentration phenomenon for a given observable `r' of a discrete time Markov chain with `μπ' as invariant ergodic measure, possibly having support on an unbounded state space. The main contribution of this paper is circumventing tedious probabilistic methods with a study of a composition of the Markov transition operator P followed by a multiplication operator defined by er. It turns out that even if the observable/ reward function is unbounded, but for some for some q>2, |er|q rightarrow 2 propto expbig(μπ(r) +frac2qq-2big)  and P is hyperbounded with norm control |P|2 rightarrow q < e^frac12[frac12-frac1q], sharp non-asymptotic concentration bounds follow. Transport-entropy inequality ensures the aforementioned upper bound on multiplication operator for all q>2. The role of reversibility in concentration phenomenon is demystified. These results are particularly useful for the reinforcement learning and controls communities as they allow for concentration inequalities w.r.t standard unbounded obersvables/reward functions where exact knowledge of the system is not available, let alone the reversibility of stationary measure.