mlsanityData to 7 Oct 2026

State Propagation Also Satisfies: A Complex-Valued State-Space Model for Deterministic State Tracking

arXiv 2608.03425 ↗First posted 4 Aug 2026cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Despite the dominance of massive language models, leading paradigms like Transformers and Mamba fundamentally falter at continuous deterministic state tracking, suffering from catastrophic out-of-distribution (OOD) collapse when generalizing to extended sequences. To shatter this bottleneck, we present the Complex State Propagator (CSP), a radically minimalist recurrent paradigm that operates strictly on the complex phase manifold without intermediate output projections, amplitude modulations, or per-step non-linearities. Crucially, we unveil an unprecedented architectural marvel: the phase signal survives extreme depth with absolute zero informational attenuation. By implementing an exact four-quadrant Coordinate-to-Phase (C-to-2) transformation governed by the (atan2(y, x)) activation, CSP forces the continuous optimization landscape to seamlessly align with discrete cyclic groups. Remarkably, with a mere 3-layer hidden topology trained on short inputs (length 16),CSP demonstrates absolute mathematical purity, achieving a nearly 100% validation accuracy and F1-score when generalized to a 4× prolonged OOD length of 64 on the canonical Mod-3 tracking task. Our work establishes complex-valued, pure-phase propagation not merely as a compact alternative, but as a dominant frontier that beats heavy networks at a fraction of their size.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 31 of its 48 references that are in the map. First posted on arXiv 4 Aug 2026.

Builds on

31 of 48 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  6. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  7. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  8. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  9. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
  10. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,283
  11. 2022Simplified State Space Layers for Sequence Modelingcited by 776
  12. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
19 more
  1. 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 599
  2. 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 594
  3. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 584
  4. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 483
  5. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 471
  6. 2022Diagonal State Spaces are as Effective as Structured State Spacescited by 350
  7. 2015Unitary Evolution Recurrent Neural Networkscited by 291
  8. 2019Theoretical Limitations of Self-Attention in Neural Sequence Modelscited by 286
  9. 2024Rotary Position Embedding for Vision Transformercited by 226
  10. 2022Exploring Length Generalization in Large Language Modelscited by 204
  11. 2024The Illusion of State in State-Space Modelscited by 181
  12. 2023Scaling Laws of RoPE-based Extrapolationcited by 124
  13. 2022Overcoming a Theoretical Limitation of Self-Attentioncited by 110
  14. 2024Unlocking State-Tracking in Linear RNNs Through Negative Eigenvaluescited by 97
  15. 2022Simplicity Bias in Transformers and their Ability to Learn Sparse Boolean Functionscited by 68
  16. 2025Test-time regression: a unifying framework for designing sequence models with associative memorycited by 63
  17. 2024Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Lengthcited by 46
  18. 2024The Expressive Capacity of State Space Models: A Formal Language Perspectivecited by 45
  19. 2024Provable Benefits of Complex Parameterizations for Structured State Space Modelscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Length-Independent State Tracking Under a Parallel Scan
  2. 2026The Expressive Limits of Diagonal SSMs for State-Tracking
  3. 2026Rethinking State Tracking in Recurrent Models Through Error Control Dynamics
  4. 2024Theoretical Foundations of Deep Selective State-Space Models
  5. 2025Log-Linear Attention
  6. 2023State-space Models with Layer-wise Nonlinearity are Universal Approximators with Exponential Decaying Memory
  7. 2026InfoMamba: An Attention-Free Hybrid Mamba-Transformer Model
  8. 2025Finite State Automata Inside Transformers with Chain-of-Thought: A Mechanistic Study on State Tracking