mlsanityData to 14 Oct 2026

Extending SSMs with the Exponentially Weighted Signature

arXiv 2603.19198 ↗First posted 19 Mar 2026stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We introduce the exponentially weighted signature (EWS), a continuous-time model that computes iterated integrals of a path, where each increment is weighted by the matrix exponential of a learnable generator over elapsed clock time. We prove that it solves a linear controlled differential equation, keeps the group-like structure and the universality of the signature, and satisfies a modified Chen identity, enabling a parallel scan. At depth one the EWS is a state-space model (SSM), and we map linear time-invariant SSMs, Mamba channels and Mamba-2 heads to it in closed form. The EWS extends SSMs through an arbitrary matrix generator, a clock that generalises the step size to causal functionals of the input, and higher truncation depths that are non-linear in the path within a single layer. Empirically, the EWS achieves the highest average accuracy and rank on six long time-series classification datasets, where depth generally helps. Learned clocks prove necessary for state tracking on formal language tasks, and at depth one, the EWS matches or exceeds competing SSMs on regression and forecasting with far fewer parameters.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 51 of its 90 references that are in the map. First posted on arXiv 19 Mar 2026.

Builds on

51 of 90 references.

References · 51

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  4. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  5. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  6. 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
  7. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
  8. 2022Simplified State Space Layers for Sequence Modelingcited by 788
  9. 2016EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfacescited by 628
  10. 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 613
  11. 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 605
  12. 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 532
38 more
  1. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 506
  2. 2022On the Parameterization and Initialization of Diagonal State Space Modelscited by 424
  3. 2023Resurrecting Recurrent Neural Networks for Long Sequencescited by 408
  4. 2024Parallelizing Linear Transformers with the Delta Rule over Sequence Lengthcited by 396
  5. 2018The UEA multivariate time series classification archive, 2018cited by 338
  6. 2024xLSTM: Extended Long Short-Term Memorycited by 327
  7. 2024Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCIcited by 327
  8. 2022On Neural Differential Equationscited by 268
  9. 2022Neural Networks and the Chomsky Hierarchycited by 217
  10. 2024CBraMod: A Criss-Cross Brain Foundation Model for EEG Decodingcited by 199
  11. 2024The Illusion of State in State-Space Modelscited by 187
  12. 2022It's Raw! Audio Generation with State-Space Modelscited by 171
  13. 2021Equinox: neural networks in JAX via callable PyTrees and filtered transformationscited by 148
  14. 2020Neural Rough Differential Equations for Long Time Seriescited by 143
  15. 2017Parallelizing Linear Recurrent Neural Nets Over Sequence Lengthcited by 127
  16. 2022How to Train Your HiPPO: State Space Models with Generalized Orthogonal Basis Projectionscited by 122
  17. 2024Unlocking State-Tracking in Linear RNNs Through Negative Eigenvaluescited by 101
  18. 2021Transformer-based Spatial-Temporal Feature Learning for EEG Decodingcited by 97
  19. 2026Mamba-3: Improved Sequence Modeling using State Space Principlescited by 89
  20. 2016Kernels for sequentially ordered datacited by 84
  21. 2019Deep Signature Transformscited by 75
  22. 2025DeltaProduct: Improving State-Tracking in Linear RNNs via Householder Productscited by 75
  23. 2024Theoretical Foundations of Deep Selective State-Space Modelscited by 74
  24. 2020The Signature Kernel is the solution of a Goursat PDEcited by 58
  25. 2023State-space Models with Layer-wise Nonlinearity are Universal Approximators with Exponential Decaying Memorycited by 39
  26. 2024Oscillatory State-Space Modelscited by 39
  27. 2023Universality of Linear Recurrences Followed by Non-linear Projections: Finite-Width Guarantees and Benefits of Complex Eigenvaluescited by 36
  28. 2024Log Neural Controlled Differential Equations: The Lie Brackets Make a Differencecited by 35
  29. 2022Signature Methods in Machine Learningcited by 34
  30. 2024Recurrent neural networks: vanishing and exploding gradients are not the end of the storycited by 25
  31. 2013Sparse arrays of signatures for online character recognitioncited by 18
  32. 2025Structured Linear CDEs: Maximally Expressive and Parallel-in-Time Sequence Modelscited by 18
  33. 2025Structured Sparse Transition Matrices to Enable State Tracking in State-Space Modelscited by 16
  34. 2025Learning to Dissipate Energy in Oscillatory State-Space Modelscited by 9
  35. 2023Nowcasting using regression on signaturescited by 5
  36. 2026The Volterra signaturecited by 5
  37. 2025Emergence of the Primacy Effect in Structured State-Space Modelscited by 4
  38. 2025Sliding-Window Signatures for Time Series: Application to Electricity Demand Forecastingcited by 1

50 most cited of 51 All 51 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Reshape and Recur: Improving SSMs with Input Reshaping and Depth Recurrence
  2. 2026The Expressive Limits of Diagonal SSMs for State-Tracking
  3. 2025Parity Requires Unified Input Dependence and Negative Eigenvalues in SSMs
  4. 2025The Curious Case of In-Training Compression of State Space Models
  5. 2024HiPPO-Prophecy: State-Space Models can Provably Learn Dynamical Systems in Context
  6. 2023Effectively Modeling Time Series with Simple Discrete State Spaces
  7. 2026LogSig-SSM: Time-Series Modelling with Multi-Scale Log-Signature Compression for State-Space Models
  8. 2026TIDES: Implicit Time-Awareness in Selective State Space Models