mlsanityData to 14 Oct 2026

Riccati State Space Models: Non-iterative Parallelization for Nonlinear Sequence Modeling

arXiv 2609.35441 ↗First posted 28 Sep 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

State space models (SSMs) achieve efficient sequence processing because their affine state updates are closed under composition and can therefore be evaluated with an associative parallel scan. Nonlinear recurrent models can provide richer, state-dependent dynamics, but generally lose this compositional structure: parallel evaluation then requires iterative methods that repeatedly linearize and scan the recurrence. We ask, what state-dependent nonlinear dynamics can be designed to remain exactly composable? We answer by introducing RiccatiSSM, a nonlinear SSM, in which each state dimension follows an input-conditioned Riccati differential equation. Its quadratic state dependence makes the local Jacobian explicitly state-dependent, while its exact per-step flow under piecewise-constant inputs is a M"obius transformation. Since M"obius maps are closed under composition and compose through 2× 2 matrix multiplication, the complete nonlinear state trajectory can be evaluated exactly with a single associative parallel scan, without iterative linearization. We further derive a constrained parameterization that ensures bounded, contractive dynamics, and avoids poles in the fractional-linear state update. Across long-sequence classification, regression, and forecasting tasks, RiccatiSSM achieves competitive predictive performance while reducing runtime by 22-33% compared to the nonlinear LrcSSM under matched architectures. These results demonstrate that state-dependent nonlinear dynamics can retain exact composability and be evaluated efficiently within a single parallel scan.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 29 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

25 of 29 references.

References · 25

  1. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  2. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  3. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  4. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  5. 2020Reformer: The Efficient Transformercited by 1,668
  6. 2017Modeling Long- and Short-Term Temporal Patterns with Deep Neural Networkscited by 879
  7. 2022Simplified State Space Layers for Sequence Modelingcited by 788
  8. 2019Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecastingcited by 607
  9. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 506
  10. 2023Resurrecting Recurrent Neural Networks for Long Sequencescited by 408
  11. 2020Liquid Time-constant Networkscited by 161
  12. 2020Neural Rough Differential Equations for Long Time Seriescited by 143
13 more
  1. 2021Closed-form Continuous-time Neural Modelscited by 65
  2. 2023Parallelizing non-linear sequential models over the sequence lengthcited by 45
  3. 2024Oscillatory State-Space Modelscited by 39
  4. 2024Towards Scalable and Stable Parallelization of Nonlinear RNNscited by 38
  5. 2024Log Neural Controlled Differential Equations: The Lie Brackets Make a Differencecited by 35
  6. 2024Rough Transformers: Lightweight and Continuous Time Series Modelling through Signature Patchingcited by 16
  7. 2025Parallelization of Non-linear State-Space Models: Scaling Up Liquid-Resistance Liquid-Capacitance Networks for Efficient Sequence Modelingcited by 10
  8. 2026Kalman Linear Attention: Parallel Bayesian Filtering For Efficient Language Modelling and State Trackingcited by 8
  9. 2024Liquid Resistance Liquid Capacitance Networkscited by 7
  10. 2025Predictability Enables Parallelization of Nonlinear State Space Modelscited by 7
  11. 2025Fixed-Point RNNs: Interpolating from Diagonal to Densecited by 5
  12. 2024Expanding Expressivity in Transformer Models with M"obiusAttentioncited by 4
  13. 2025A Unifying Framework for Parallelizing Sequential Models with Linear Dynamical Systemscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025The Curious Case of In-Training Compression of State Space Models
  2. 2025MS-SSM: A Multi-Scale State Space Model for Efficient Sequence Modeling
  3. 2023Spectral State Space Models
  4. 2024HiPPO-Prophecy: State-Space Models can Provably Learn Dynamical Systems in Context
  5. 2023State-space Models with Layer-wise Nonlinearity are Universal Approximators with Exponential Decaying Memory
  6. 2026MaRK: Markov-adapted Recurrent Kernels for Dynamic Operator Conditioning in State Space Models
  7. 2025Advancing Intelligent Sequence Modeling: Evolution, Trade-offs, and Applications of State-Space Architectures from S4 to Mamba
  8. 2024HOPE for a Robust Parameterization of Long-memory State Space Models