What Can a Recurrent State Safely Forget?
Not on the map. Why?
Abstract (arXiv)
Recurrent models must preserve information that changes future behavior while suppressing hidden-state error. These objectives conflict: contraction improves stability, but contraction along a future-distinguishing direction destroys memory. We formalize this boundary through the predictive quotient of a recurrent state space. Two hidden states are equivalent when they induce the same conditional future; their equivalence classes form predictive fibers. Every exact semantics-preserving corrector acts as the identity on this quotient. At a regular point with hidden dimension d and predictive dimension k, it can eliminate at most d - k independent directions. This establishes a discrete-continuous boundary: finite predictive states admit positive-radius exact correction basins, whereas an uncountable continuum of future-distinguishable states cannot be decoded after arbitrary positive-radius perturbations in finite-dimensional Euclidean space. To operationalize this principle, we develop an auditable finite-future framework. A compact deployment bank W is evaluated against an independent audit bank A (W subseteq A) on a declared correction domain. Under generative probe access and audit-metric coverage, finite stochastic rollouts furnish a high-probability certificate for the separation margin Omega_{W|A}(delta). Preserving learned W-predictions within this certified margin guarantees bounded audit-semantic distortion. For intrinsic audit dimension k, the required probe outcomes scale as O(M * Omega^{-(k+2)}), where M = |A|; a matching minimax lower bound proves this exponent is optimal. Extending guarantees to continuous futures is achieved via an explicit completeness modulus. Controlled experiments validate the certified margins, scaling laws, and automated probe refinement under a safety-first evaluation paradigm.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 5 of its 14 references that are in the map. First posted on arXiv 20 Sep 2026.
Builds on
5 of 14 references.
References · 5
- 2018Stable Recurrent Modelscited by 65
- 2017Predictive State Recurrent Neural Networkscited by 14
- 2017Predictive-State Decoders: Encoding the Future into Recurrent Networkscited by 11
- 2024Back to the Continuous Attractorcited by 6
- 2023Data-Driven Observability Analysis for Nonlinear Stochastic Systemscited by 1
Built on by
Not cited yet.