mlsanityData to 7 Oct 2026

Non-Commutative State Tracking with Input-Dependent Low-Rank Updates in Mamba-3

arXiv 2609.28273 ↗First posted 23 Sep 2026eess.SY cs.LG cs.SYcited by 0

On the map: State-space sequence models (S4, Mamba) → Core SSM architectures for language

Abstract (arXiv)

State tracking from sequential observations can require both retaining information and updating it by composing observed operations. We extend Mamba-3's diagonal transition with an input-dependent low-rank reflection term to support noncommutative state tracking, in which the order of operations matters. The rank-one update couples state coordinates along an input-dependent direction, enabling non-diagonal state transitions within a single Mamba-3 block. The extension preserves Mamba-3's exponential-trapezoidal discretization, rotary embeddings (RoPE), and readout. For training, we adapt chunkwise computation to parallelize the proposed recurrence within each chunk. Experiments cover group word problems with discrete inputs and a shell game with continuous observations, in which a policy is trained by behavioral cloning. Among the models selected for their strong performance under fixed timing, the proposed model maintains higher tracking success on longer swap sequences in the shell game with continuous observations and timing jitter. These experiments show that the proposed method achieves high accuracy on the evaluated non-commutative tracking tasks, improving on standard Mamba-3. The extension thus offers a Mamba-3-based approach to non-commutative state tracking.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 21 of its 26 references that are in the map. First posted on arXiv 23 Sep 2026.

Builds on

21 of 26 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026Mamba-3: Improved Sequence Modeling using State Space Principles16 Mar
  2. 2024Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues19 Nov
  3. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality31 May
  4. 2024Theoretical Foundations of Deep Selective State-Space Models29 Feb
  5. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spaces1 Dec
  6. 2022Diagonal State Spaces are as Effective as Structured State Spaces27 Mar
  7. 2021Efficiently Modeling Long Sequences with Structured State Spaces31 Oct

Other references · 14

  1. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  2. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  3. 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 600
  4. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 483
  5. 2022On the Parameterization and Initialization of Diagonal State Space Modelscited by 418
  6. 2024Parallelizing Linear Transformers with the Delta Rule over Sequence Lengthcited by 373
  7. 2022Transformers Learn Shortcuts to Automatacited by 260
  8. 2024The Illusion of State in State-Space Modelscited by 181
  9. 2025RWKV-7 "Goose" with Expressive Dynamic State Evolutioncited by 134
  10. 2025DeltaProduct: Improving State-Tracking in Linear RNNs via Householder Productscited by 69
  11. 2025(How) Do Language Models Track State?cited by 31
  12. 2025MTIL: Encoding Full History with Mamba for Temporal Imitation Learningcited by 10
2 more
  1. 2025RoboSSM: Scalable In-context Imitation Learning via State-Space Modelscited by 5
  2. 2026Dissecting Linear Recurrent Models: How Different Gating Strategies Drive Selectivity and Generalizationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026A Held-Out Transition-Pair Falsifier for Long-Horizon Non-Abelian State Tracking
  2. 2025Block-Biased Mamba for Long-Range Sequence Processing
  3. 2026Naju: A Native Discrete State-Space Model with Independent Retention and Writing for Long-Sequence Memory
  4. 2026Length-Independent State Tracking Under a Parallel Scan
  5. 2026Mamba-Assisted Non-Markovian Closure for Reduced-Order Modeling
  6. 2026The Expressive Limits of Diagonal SSMs for State-Tracking
  7. 2024Sparse Mamba: Introducing Controllability, Observability, And Stability To Structural State Space Models
  8. 2026Rethinking State Tracking in Recurrent Models Through Error Control Dynamics