# Auditing Finite-Lag Statistics of Recurrent Representations

arXiv: https://arxiv.org/abs/2607.01746 · HTML: https://mlsanity.com/paper/2607.01746/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 43 references that are in the map.
- First posted on arXiv 2 Jul 2026.

## Builds on (most cited first, 17 of 17)

- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Prevalence of Neural Collapse during the terminal phase of deep learning training](https://mlsanity.com/paper/2008.08186/) (2020)
- [Intrinsic dimension of data representations in deep neural networks](https://mlsanity.com/paper/1905.12784/) (2019)
- [VAMPnets: Deep learning of molecular kinetics](https://mlsanity.com/paper/1710.06012/) (2017)
- [Universality and individuality in neural dynamics across large populations of recurrent networks](https://mlsanity.com/paper/1907.08549/) (2019)
- [Variational approach for learning Markov processes from time series data](https://mlsanity.com/paper/1707.04659/) (2017)
- [Linking connectivity, dynamics and computations in low-rank recurrent neural networks](https://mlsanity.com/paper/1711.09672/) (2017)
- [Eigendecompositions of Transfer Operators in Reproducing Kernel Hilbert Spaces](https://mlsanity.com/paper/1712.01572/) (2017)
- [Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics](https://mlsanity.com/paper/1906.10720/) (2019)
- [Lipschitz regularity of graph Laplacians on random data clouds](https://mlsanity.com/paper/2007.06679/) (2020)
- [Reverse engineering recurrent neural networks with Jacobian switching linear dynamical systems](https://mlsanity.com/paper/2111.01256/) (2021)
- [Assessing Neural Network Representations During Training Using Noise-Resilient Diffusion Spectral Entropy](https://mlsanity.com/paper/2312.04823/) (2023)
- [Shaping dynamics with multiple populations in low-rank recurrent networks](https://mlsanity.com/paper/2007.02062/) (2020)
- [Hidden Traveling Waves bind Working Memory Variables in Recurrent Neural Networks](https://mlsanity.com/paper/2402.10163/) (2024)
- [Dynamical phases of short-term memory mechanisms in RNNs](https://mlsanity.com/paper/2502.17433/) (2025)
- [Diffusion Operator Geometry of Feedforward Representations](https://mlsanity.com/paper/2605.01107/) (2026)

## Abstract (arXiv)

Hidden-state trajectories are computational traces that record how recurrent representations evolve as networks process sequences. When exact state repeats are unavailable, one way to estimate conditional motion is to borrow successors from nearby states, yielding a finite-bandwidth successor smoother. However, this procedure can change the displacement being measured. At a median-distance bandwidth, write-phase displacement energy is about 1.88× the exact value for GRU and LSTM. Elman instead shows near agreement in total energy despite large, opposing finite-bandwidth contributions. We therefore compute the conditional-mean energy fraction exactly on repeat-copy. Training reverses the architecture ordering of this fraction, with GRU and LSTM rising to about 94% while Elman falls to about 55%. The centered remainder is small in energy but can still expose input information. For LSTM, linear four-bit decoding rises from 0.804 on total displacement to 1.000 after exact conditional centering.