# Teacher Forcing as Generalized Bayes: Optimization Geometry Mismatch in Switching Surrogates for Chaotic Dynamics

arXiv: https://arxiv.org/abs/2604.25904 · HTML: https://mlsanity.com/paper/2604.25904/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 31 references that are in the map.
- First posted on arXiv 28 Apr 2026.

## Builds on (most cited first, 13 of 13)

- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [A General Framework for Updating Belief Distributions](https://mlsanity.com/paper/1306.6430/) (2013)
- [On the difficulty of learning chaotic dynamics with RNNs](https://mlsanity.com/paper/2110.07238/) (2021)
- [Generalized Teacher Forcing for Learning Chaotic Dynamics](https://mlsanity.com/paper/2306.04406/) (2023)
- [Rao-Blackwellised Particle Filtering for Dynamic Bayesian Networks](https://mlsanity.com/paper/1301.3853/) (2013)
- [Tractable Dendritic RNNs for Reconstructing Nonlinear Dynamical Systems](https://mlsanity.com/paper/2207.02542/) (2022)
- [Out-of-Domain Generalization in Dynamical Systems Reconstruction](https://mlsanity.com/paper/2402.18377/) (2024)
- [DySLIM: Dynamics Stable Learning by Invariant Measure for Chaotic Systems](https://mlsanity.com/paper/2402.04467/) (2024)
- [Almost-Linear RNNs Yield Highly Interpretable Symbolic Codes in Dynamical Systems Reconstruction](https://mlsanity.com/paper/2410.14240/) (2024)
- [Bifurcations and loss jumps in RNN training](https://mlsanity.com/paper/2310.17561/) (2023)
- [When are dynamical systems learned from time series data statistically accurate?](https://mlsanity.com/paper/2411.06311/) (2024)
- [Uncovering the Computational Roles of Nonlinearity in Sequence Modeling Using Almost-Linear RNNs](https://mlsanity.com/paper/2506.07919/) (2025)
- [Detecting Invariant Manifolds in ReLU-Based RNNs](https://mlsanity.com/paper/2510.03814/) (2025)

## Built on by (most cited first, 2 of 2)

- [The Dynamic-Probabilistic Consistency Gap in Chaotic Surrogate Modeling](https://mlsanity.com/paper/2605.31547/) (2026)
- [Correct then Forecast: Observer State-Space Models for Time Series Forecasting](https://mlsanity.com/paper/2609.33566/) (2026)

## Abstract (arXiv)

Identity teacher forcing (ITF) enables stable training of deterministic recurrent surrogates for chaotic dynamical systems and has been highly effective for dynamical systems reconstruction (DSR) with recurrent neural networks (RNNs), including interpretable almost-linear RNNs (AL-RNNs). However, as an intervention-based prediction loss (and thus a generalized Bayes update), teacher forcing need not match the free-running model's marginal likelihood geometry. We compare the objective-induced curvatures of ITF and marginal likelihood in a probabilistic switching augmentation of AL-RNNs, estimating ambiguity-aware observed information via Louis' identity. In the switching setting studied here, conditioning on a single forced regime path (as ITF does) inflates curvature, while marginal likelihood curvature is reduced by a missing-information correction when multiple switching explanations remain plausible. In Lorenz-63 experiments, windowed evidence fine-tuning improves held-out evidence but can degrade dynamical quantities of interest (QoIs) relative to ITF-pretrained models.