mlsanityData to 14 Oct 2026

Deep Learning for Markov Chains: Lyapunov Functions, Poisson's Equation, and Stationary Distributions

arXiv 2508.16737 ↗First posted 22 Aug 2025cs.LG math.PRcited by 2

Not on the map. Why?

Abstract (arXiv)

Lyapunov functions are fundamental to establishing the stability of Markovian models, yet their construction typically demands substantial creativity and analytical effort. In this paper, we show that deep learning can automate this process by training neural networks to satisfy integral equations derived from first-transition analysis. Beyond stability analysis, our approach can be adapted to solve Poisson's equation and estimate stationary distributions. While neural networks are inherently function approximators on compact domains, it turns out that our approach remains effective when applied to Markov chains on non-compact state spaces. We demonstrate the effectiveness of this methodology through several examples from queueing theory and beyond.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 12 of its 36 references that are in the map. First posted on arXiv 22 Aug 2025.

Builds on

12 of 36 references.

References · 12

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2016Error bounds for approximations with deep ReLU networkscited by 574
  4. 2017Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networkscited by 311
  5. 2018Optimal approximation of continuous functions by very deep ReLU networkscited by 166
  6. 2018Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionalitycited by 162
  7. 2022Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spacescited by 51
  8. 2023Physics-Informed Neural Network Lyapunov Functions: PDE Characterization, Learning, and Verificationcited by 20
  9. 2020Approximating smooth functions by deep neural networks with sigmoid activation functioncited by 18
  10. 2024The surprising efficiency of temporal difference learning for rare event predictioncited by 7
  11. 2022Constructing unbiased gradient estimators with finite variance for conditional stochastic optimizationcited by 6
  12. 2024Deep Learning for Computing Convergence Rates of Markov Chainscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Deep Learning Method for Stationary Distribution of Reflected Brownian Motioncited by 0
  2. 2026Machine learning kinetics from molecular dynamics datacited by 0