# Stochastic approximation in non-markovian environments revisited

arXiv: https://arxiv.org/abs/2603.21091 · HTML: https://mlsanity.com/paper/2603.21091/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 3 of its 13 references that are in the map.
- First posted on arXiv 22 Mar 2026.

## Builds on (most cited first, 3 of 3)

- [Reinforcement Learning in Non-Markovian Environments](https://mlsanity.com/paper/2211.01595/) (2022)
- [Training Dynamics of Softmax Self-Attention: Fast Global Convergence via Preconditioning](https://mlsanity.com/paper/2603.01514/) (2026)
- [Controlled Martingale Problems And Their Markov Mimics](https://mlsanity.com/paper/2309.00488/) (2023)

## Abstract (arXiv)

Based on some recent work of the author on stochastic approximation in non-markovian environments, the situation when the driving random process is non-ergodic in addition to being non-markovian is considered. Using this, we propose an analytic framework for understanding transformer based learning, specifically, the `attention' mechanism, and continual learning, both of which depend on the entire past in principle.