mlsanityData to 7 Oct 2026

Parallelization of Non-linear State-Space Models: Scaling Up Liquid-Resistance Liquid-Capacitance Networks for Efficient Sequence Modeling

arXiv 2505.21717 ↗First posted 27 May 2025cs.LG cs.AI cs.NEcited by 10

On the map: State-space sequence models (S4, Mamba) → Core SSM architectures for language

Abstract (arXiv)

We present LrcSSM, a textitnon-linear recurrent model that processes long sequences as fast as today's linear state-space layers. By forcing its Jacobian matrix to be diagonal, the full sequence can be solved in parallel, giving O(TD) computational work and memory and only O(log T) sequential depth, for input-sequence length T and a state dimension D. Moreover, LrcSSM offers a formal gradient-stability guarantee that other input-varying systems such as Liquid-S4 and Mamba do not provide. Importantly, the diagonal Jacobian structure of our model results in no performance loss compared to the original model with dense Jacobian, and the approach can be generalized to other non-linear recurrent models, demonstrating broader applicability. On a suite of long-range forecasting tasks, we demonstrate that LrcSSM outperforms Transformers, LRU, S5, and Mamba.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 29 of its 46 references that are in the map. First posted on arXiv 27 May 2025.

Builds on

29 of 46 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues19 Nov
  2. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality31 May
  3. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spaces1 Dec
  4. 2022Simplified State Space Layers for Sequence Modeling9 Aug
  5. 2021Efficiently Modeling Long Sequences with Structured State Spaces31 Oct

Other references · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Scaling Laws for Neural Language Modelscited by 6,411
  3. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  4. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  5. 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 600
  6. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 501
  7. 2022On the Parameterization and Initialization of Diagonal State Space Modelscited by 418
  8. 2023Resurrecting Recurrent Neural Networks for Long Sequencescited by 397
  9. 2023Hyena Hierarchy: Towards Larger Convolutional Language Modelscited by 388
  10. 2020Liquid Time-constant Networkscited by 160
  11. 2020Neural Rough Differential Equations for Long Time Seriescited by 140
  12. 2022Liquid Structural State-Space Modelscited by 134
12 more
  1. 2017Parallelizing Linear Recurrent Neural Nets Over Sequence Lengthcited by 126
  2. 2024Mechanistic Design and Scaling of Hybrid Architecturescited by 57
  3. 2023Parallelizing non-linear sequential models over the sequence lengthcited by 45
  4. 2016Minimal Gated Unit for Recurrent Neural Networkscited by 40
  5. 2024Towards Scalable and Stable Parallelization of Nonlinear RNNscited by 38
  6. 2024Oscillatory State-Space Modelscited by 35
  7. 2024Log Neural Controlled Differential Equations: The Lie Brackets Make a Differencecited by 34
  8. 2023FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Corescited by 32
  9. 2024State-Free Inference of State-Space Models: The Transfer Function Approachcited by 18
  10. 2024Rough Transformers: Lightweight and Continuous Time Series Modelling through Signature Patchingcited by 16
  11. 2024Liquid Resistance Liquid Capacitance Networkscited by 7
  12. 2023Learning with Chemical versus Electrical Synapses -- Does it Make a Difference?cited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 6.

10 papers.

  1. 2025ParaRNN: Unlocking Parallel Training of Nonlinear RNNs for Large Language Modelscited by 13
  2. 2025Predictability Enables Parallelization of Nonlinear State Space Modelscited by 7
  3. 2025A Unifying Framework for Parallelizing Sequential Models with Linear Dynamical Systemscited by 4
  4. 2025Context-Selective State Space Models: Feedback is All You Needcited by 1
  5. 2026Adaptive Control in Autonomous Driving via Real-Time Recurrent RLcited by 0
  6. 2026Synaptic Activation and Dual Liquid Dynamics for Interpretable Bio-Inspired Modelscited by 0
  7. 2026Reshape and Recur: Improving SSMs with Input Reshaping and Depth Recurrencecited by 0
  8. 2026Faithful Embeddings of Irregular and Asynchronous Data for Online Log-NCDEscited by 0
  9. 2026Riccati State Space Models: Non-iterative Parallelization for Nonlinear Sequence Modelingcited by 0
  10. 2026Switching Linear Attentioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026MaRK: Markov-adapted Recurrent Kernels for Dynamic Operator Conditioning in State Space Models
  2. 2025Echo State Networks as State-Space Models: A Systems Perspective
  3. 2023Effectively Modeling Time Series with Simple Discrete State Spaces
  4. 2026Improved state mixing in higher-order and block diagonal linear recurrent networks
  5. 2024Recurrent neural networks: vanishing and exploding gradients are not the end of the story
  6. 2023State-space Models with Layer-wise Nonlinearity are Universal Approximators with Exponential Decaying Memory
  7. 2024State-Space Modeling in Long Sequence Processing: A Survey on Recurrence in the Transformer Era
  8. 2026DSSMs: State Space Models with Explicit Memory via Delay Differential Equations