mlsanityData to 14 Oct 2026

A Held-Out Transition-Pair Falsifier for Long-Horizon Non-Abelian State Tracking

arXiv 2606.07254 ↗First posted 5 Jun 2026cs.LG cs.FLcited by 3

Not on the map. Why?

Abstract (arXiv)

State tracking exposes a sharp limitation of sequence models: the relevant signal is often not a summary of observed tokens, but an ordered latent state that evolves through non-commutative transformations. We introduce a held-out transition-pair falsifier for finite non-Abelian group tracking. The protocol forbids selected ordered generator pairs during training and requires the same local patterns during evaluation, blocking one direct local-transition memorization pathway. In a controlled S3 × S3 benchmark, a projected recurrent state model trained only on length-8 sequences produces error-free final-state predictions (perfect 250/250 per horizon) through evaluation horizons up to 1,048,576 tokens across five seeds. Matched native-readout baselines, including bag, GRU, and a single-configuration structured state-space model, remain near floor under the same protocol. Projection-matched GRU, structured SSM, and bag baselines equipped with analogous finite-group prototype readouts also remain near chance under the same split. Mechanism diagnostics show that hard projection coincides with low homomorphism error, low state-consistency drift, and non-trivial commutator separation, while softened projection collapses final-state accuracy. Clean-split audits verify zero verbatim reduced-word overlap and zero structural-template overlap between training and evaluation partitions. The evidence is scoped to this controlled finite-group falsifier rather than to a general architecture ranking. Within that regime, explicit projected non-commutative state composition acts as a useful inductive bias for long-horizon hidden-state tracking.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 4 of its 8 references that are in the map. First posted on arXiv 5 Jun 2026.

Builds on

4 of 8 references.

References · 4

  1. 2024The Illusion of State in State-Space Modelscited by 187
  2. 2025Structured Sparse Transition Matrices to Enable State Tracking in State-Space Modelscited by 16
  3. 2026The Expressive Limits of Diagonal SSMs for State-Trackingcited by 9
  4. 2026On the "Induction Bias" in Sequence Modelscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Rethinking State Tracking in Recurrent Models Through Error Control Dynamics
  2. 2026Length-Independent State Tracking Under a Parallel Scan
  3. 2026Long-Horizon State Tracking in LLMs: Executing MD5 through a Deep Sequence of Dependent Tool Calls
  4. 2026Non-Commutative State Tracking with Input-Dependent Low-Rank Updates in Mamba-3
  5. 2026Naju: A Native Discrete State-Space Model with Independent Retention and Writing for Long-Sequence Memory
  6. 2025Scaling Laws for State Dynamics in Large Language Models
  7. 2026State Propagation Also Satisfies: A Complex-Valued State-Space Model for Deterministic State Tracking
  8. 2025Finite State Automata Inside Transformers with Chain-of-Thought: A Mechanistic Study on State Tracking