# A Diagonal Structured State Space Model on Loihi 2 for Efficient Streaming Sequence Processing

arXiv: https://arxiv.org/abs/2409.15022 · HTML: https://mlsanity.com/paper/2409.15022/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 34 of its 49 references that are in the map.
- First posted on arXiv 23 Sep 2024.

## Placed in

- State-space sequence models (S4, Mamba) / [Core SSM architectures for language](https://mlsanity.com/ssm/core-ssm-architectures-for-language/)

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2208.04933/) (2022)
- [VideoBERT: A Joint Model for Video and Language Representation Learning](https://mlsanity.com/paper/1904.01766/) (2019)
- [Long Range Arena: A Benchmark for Efficient Transformers](https://mlsanity.com/paper/2011.04006/) (2020)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers](https://mlsanity.com/paper/2110.13985/) (2021)
- [HiPPO: Recurrent Memory with Optimal Polynomial Projections](https://mlsanity.com/paper/2008.07669/) (2020)
- [Efficiently Scaling Transformer Inference](https://mlsanity.com/paper/2211.05102/) (2022)
- [On the Parameterization and Initialization of Diagonal State Space Models](https://mlsanity.com/paper/2206.11893/) (2022)
- [Diagonal State Spaces are as Effective as Structured State Spaces](https://mlsanity.com/paper/2203.14343/) (2022)
- [The Hardware Lottery](https://mlsanity.com/paper/2009.06489/) (2020)
- [Liquid Structural State-Space Models](https://mlsanity.com/paper/2209.12951/) (2022)
- [AI and Memory Wall](https://mlsanity.com/paper/2403.14123/) (2024)
- [Lipschitz Recurrent Neural Networks](https://mlsanity.com/paper/2006.12070/) (2020)
- [SpikeGPT: Generative Pre-trained Language Model with Spiking Neural Networks](https://mlsanity.com/paper/2302.13939/) (2023)
- [Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challenges](https://mlsanity.com/paper/2404.16112/) (2024)
- [Learning Longer-term Dependencies in RNNs with Auxiliary Losses](https://mlsanity.com/paper/1803.00144/) (2018)
- [A Long Short-Term Memory for AI Applications in Spike-based Neuromorphic Hardware](https://mlsanity.com/paper/2107.03992/) (2021)
- [Efficient Video and Audio processing with Loihi 2](https://mlsanity.com/paper/2310.03251/) (2023)
- [Green Edge AI: A Contemporary Survey](https://mlsanity.com/paper/2312.00333/) (2023)
- [Spiking Structured State Space Model for Monaural Speech Enhancement](https://mlsanity.com/paper/2309.03641/) (2023)
- [DYNAP-SE2: a scalable multi-core dynamic neuromorphic asynchronous spiking neural network processor](https://mlsanity.com/paper/2310.00564/) (2023)
- [SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Models](https://mlsanity.com/paper/2408.14909/) (2024)

## Built on by (most cited first, 6 of 6)

- [State-Space Model Inspired Multiple-Input Multiple-Output Spiking Neurons](https://mlsanity.com/paper/2504.02591/) (2025)
- [Accelerating Linear Recurrent Neural Networks for the Edge with Unstructured Sparsity](https://mlsanity.com/paper/2502.01330/) (2025)
- [Quantizing Small-Scale State-Space Models for Edge AI](https://mlsanity.com/paper/2506.12480/) (2025)
- [Low-Bit Data Processing Using Multiple-Output Spiking Neurons with Non-linear Reset Feedback](https://mlsanity.com/paper/2508.06292/) (2025)
- [QS4D: Quantization-aware training for efficient hardware deployment of structured state-space sequential models](https://mlsanity.com/paper/2507.06079/) (2025)
- [Federated Learning of Spiking Neural Networks under Heterogeneous Temporal Resolutions](https://mlsanity.com/paper/2605.15355/) (2026)

## Abstract (arXiv)

Deep State-Space Models (SSM) demonstrate state-of-the art performance on long-range sequence modeling tasks. While the recurrent structure of SSMs can be efficiently implemented as a convolution or as a parallel scan during training, recurrent token-by-token processing cannot currently be implemented efficiently on GPUs. Here, we demonstrate efficient token-by-token inference of the SSM S4D on Intel's Loihi 2 state-of-the-art neuromorphic processor. We compare this first ever neuromorphic-hardware implementation of an SSM on sMNIST, psMNIST, and sCIFAR to a recurrent and a convolutional implementation of S4D on Jetson Orin Nano (Jetson). While we find Jetson to perform better in an offline sample-by-sample based batched processing mode, Loihi 2 outperforms during token-by-token based processing, where it consumes 1000 times less energy with a 75 times lower latency and a 75 times higher throughput compared to the recurrent implementation of S4D on Jetson. This opens up new avenues towards efficient real-time streaming applications of SSMs.