Extending SSMs with the Exponentially Weighted Signature
Not on the map. Why?
Abstract (arXiv)
We introduce the exponentially weighted signature (EWS), a continuous-time model that computes iterated integrals of a path, where each increment is weighted by the matrix exponential of a learnable generator over elapsed clock time. We prove that it solves a linear controlled differential equation, keeps the group-like structure and the universality of the signature, and satisfies a modified Chen identity, enabling a parallel scan. At depth one the EWS is a state-space model (SSM), and we map linear time-invariant SSMs, Mamba channels and Mamba-2 heads to it in closed form. The EWS extends SSMs through an arbitrary matrix generator, a clock that generalises the step size to causal functionals of the input, and higher truncation depths that are non-linear in the path within a single layer. Empirically, the EWS achieves the highest average accuracy and rank on six long time-series classification datasets, where depth generally helps. Learned clocks prove necessary for state tracking on formal language tasks, and at depth one, the EWS matches or exceeds competing SSMs on regression and forecasting with far fewer parameters.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 51 of its 90 references that are in the map. First posted on arXiv 19 Mar 2026.
Builds on
51 of 90 references.
References · 51
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
- 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
- 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
- 2020Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecastingcited by 2,169
- 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
- 2022Simplified State Space Layers for Sequence Modelingcited by 788
- 2016EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfacescited by 628
- 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 613
- 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 605
- 2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 532
38 more
- 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 506
- 2022On the Parameterization and Initialization of Diagonal State Space Modelscited by 424
- 2023Resurrecting Recurrent Neural Networks for Long Sequencescited by 408
- 2024Parallelizing Linear Transformers with the Delta Rule over Sequence Lengthcited by 396
- 2018The UEA multivariate time series classification archive, 2018cited by 338
- 2024xLSTM: Extended Long Short-Term Memorycited by 327
- 2024Large Brain Model for Learning Generic Representations with Tremendous EEG Data in BCIcited by 327
- 2022On Neural Differential Equationscited by 268
- 2022Neural Networks and the Chomsky Hierarchycited by 217
- 2024CBraMod: A Criss-Cross Brain Foundation Model for EEG Decodingcited by 199
- 2024The Illusion of State in State-Space Modelscited by 187
- 2022It's Raw! Audio Generation with State-Space Modelscited by 171
- 2021Equinox: neural networks in JAX via callable PyTrees and filtered transformationscited by 148
- 2020Neural Rough Differential Equations for Long Time Seriescited by 143
- 2017Parallelizing Linear Recurrent Neural Nets Over Sequence Lengthcited by 127
- 2022How to Train Your HiPPO: State Space Models with Generalized Orthogonal Basis Projectionscited by 122
- 2024Unlocking State-Tracking in Linear RNNs Through Negative Eigenvaluescited by 101
- 2021Transformer-based Spatial-Temporal Feature Learning for EEG Decodingcited by 97
- 2026Mamba-3: Improved Sequence Modeling using State Space Principlescited by 89
- 2016Kernels for sequentially ordered datacited by 84
- 2019Deep Signature Transformscited by 75
- 2025DeltaProduct: Improving State-Tracking in Linear RNNs via Householder Productscited by 75
- 2024Theoretical Foundations of Deep Selective State-Space Modelscited by 74
- 2020The Signature Kernel is the solution of a Goursat PDEcited by 58
- 2023State-space Models with Layer-wise Nonlinearity are Universal Approximators with Exponential Decaying Memorycited by 39
- 2024Oscillatory State-Space Modelscited by 39
- 2023Universality of Linear Recurrences Followed by Non-linear Projections: Finite-Width Guarantees and Benefits of Complex Eigenvaluescited by 36
- 2024Log Neural Controlled Differential Equations: The Lie Brackets Make a Differencecited by 35
- 2022Signature Methods in Machine Learningcited by 34
- 2024Recurrent neural networks: vanishing and exploding gradients are not the end of the storycited by 25
- 2013Sparse arrays of signatures for online character recognitioncited by 18
- 2025Structured Linear CDEs: Maximally Expressive and Parallel-in-Time Sequence Modelscited by 18
- 2025Structured Sparse Transition Matrices to Enable State Tracking in State-Space Modelscited by 16
- 2025Learning to Dissipate Energy in Oscillatory State-Space Modelscited by 9
- 2023Nowcasting using regression on signaturescited by 5
- 2026The Volterra signaturecited by 5
- 2025Emergence of the Primacy Effect in Structured State-Space Modelscited by 4
- 2025Sliding-Window Signatures for Time Series: Application to Electricity Demand Forecastingcited by 1
50 most cited of 51 All 51 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Reshape and Recur: Improving SSMs with Input Reshaping and Depth Recurrence
- 2026The Expressive Limits of Diagonal SSMs for State-Tracking
- 2025Parity Requires Unified Input Dependence and Negative Eigenvalues in SSMs
- 2025The Curious Case of In-Training Compression of State Space Models
- 2024HiPPO-Prophecy: State-Space Models can Provably Learn Dynamical Systems in Context
- 2023Effectively Modeling Time Series with Simple Discrete State Spaces
- 2026LogSig-SSM: Time-Series Modelling with Multi-Scale Log-Signature Compression for State-Space Models
- 2026TIDES: Implicit Time-Awareness in Selective State Space Models