Non-Commutative State Tracking with Input-Dependent Low-Rank Updates in Mamba-3
On the map: State-space sequence models (S4, Mamba) → Core SSM architectures for language
Abstract (arXiv)
State tracking from sequential observations can require both retaining information and updating it by composing observed operations. We extend Mamba-3's diagonal transition with an input-dependent low-rank reflection term to support noncommutative state tracking, in which the order of operations matters. The rank-one update couples state coordinates along an input-dependent direction, enabling non-diagonal state transitions within a single Mamba-3 block. The extension preserves Mamba-3's exponential-trapezoidal discretization, rotary embeddings (RoPE), and readout. For training, we adapt chunkwise computation to parallelize the proposed recurrence within each chunk. Experiments cover group word problems with discrete inputs and a shell game with continuous observations, in which a policy is trained by behavioral cloning. Among the models selected for their strong performance under fixed timing, the proposed model maintains higher tracking success on longer swap sequences in the shell game with continuous observations and timing jitter. These experiments show that the proposed method achieves high accuracy on the evaluated non-commutative tracking tasks, improving on standard Mamba-3. The extension thus offers a Mamba-3-based approach to non-commutative state tracking.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 21 of its 26 references that are in the map. First posted on arXiv 23 Sep 2026.
Builds on
21 of 26 references.
Closest ancestors · 7
Same direction, newest first
- 2026Mamba-3: Improved Sequence Modeling using State Space Principles16 Mar
- 2024Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues19 Nov
- 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality31 May
- 2024Theoretical Foundations of Deep Selective State-Space Models29 Feb
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spaces1 Dec
- 2022Diagonal State Spaces are as Effective as Structured State Spaces27 Mar
- 2021Efficiently Modeling Long Sequences with Structured State Spaces31 Oct
Other references · 14
- 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
- 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
- 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 600
- 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 483
- 2022On the Parameterization and Initialization of Diagonal State Space Modelscited by 418
- 2024Parallelizing Linear Transformers with the Delta Rule over Sequence Lengthcited by 373
- 2022Transformers Learn Shortcuts to Automatacited by 260
- 2024The Illusion of State in State-Space Modelscited by 181
- 2025RWKV-7 "Goose" with Expressive Dynamic State Evolutioncited by 134
- 2025DeltaProduct: Improving State-Tracking in Linear RNNs via Householder Productscited by 69
- 2025(How) Do Language Models Track State?cited by 31
- 2025MTIL: Encoding Full History with Mamba for Temporal Imitation Learningcited by 10
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026A Held-Out Transition-Pair Falsifier for Long-Horizon Non-Abelian State Tracking
- 2025Block-Biased Mamba for Long-Range Sequence Processing
- 2026Naju: A Native Discrete State-Space Model with Independent Retention and Writing for Long-Sequence Memory
- 2026Length-Independent State Tracking Under a Parallel Scan
- 2026Mamba-Assisted Non-Markovian Closure for Reduced-Order Modeling
- 2026The Expressive Limits of Diagonal SSMs for State-Tracking
- 2024Sparse Mamba: Introducing Controllability, Observability, And Stability To Structural State Space Models
- 2026Rethinking State Tracking in Recurrent Models Through Error Control Dynamics