# QS4D: Quantization-aware training for efficient hardware deployment of structured state-space sequential models

arXiv: https://arxiv.org/abs/2507.06079 · HTML: https://mlsanity.com/paper/2507.06079/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 21 of its 35 references that are in the map.
- First posted on arXiv 8 Jul 2025.

## Placed in

- State-space sequence models (S4, Mamba) / [Core SSM architectures for language](https://mlsanity.com/ssm/core-ssm-architectures-for-language/)

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2208.04933/) (2022)
- [Long Range Arena: A Benchmark for Efficient Transformers](https://mlsanity.com/paper/2011.04006/) (2020)
- [HiPPO: Recurrent Memory with Optimal Polynomial Projections](https://mlsanity.com/paper/2008.07669/) (2020)
- [On the Parameterization and Initialization of Diagonal State Space Models](https://mlsanity.com/paper/2206.11893/) (2022)
- [The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits](https://mlsanity.com/paper/2402.17764/) (2024)
- [The Heidelberg spiking datasets for the systematic evaluation of spiking neural networks](https://mlsanity.com/paper/1910.07407/) (2019)
- [Scaling Laws for Precision](https://mlsanity.com/paper/2411.04330/) (2024)
- [PokeBNN: A Binary Pursuit of Lightweight Accuracy](https://mlsanity.com/paper/2112.00133/) (2021)
- [Quamba: A Post-Training Quantization Recipe for Selective State Space Models](https://mlsanity.com/paper/2410.13229/) (2024)
- [Recurrent Neural Networks With Limited Numerical Precision](https://mlsanity.com/paper/1608.06902/) (2016)
- [DeepHeart: Semi-Supervised Sequence Learning for Cardiovascular Risk Prediction](https://mlsanity.com/paper/1802.02511/) (2018)
- [Bi-Mamba: Towards Accurate 1-Bit State Space Models](https://mlsanity.com/paper/2411.11843/) (2024)
- [Q-S5: Towards Quantized State Space Models](https://mlsanity.com/paper/2406.09477/) (2024)
- [Hardware Aware Training for Efficient Keyword Spotting on General Purpose and Specialized Hardware](https://mlsanity.com/paper/2009.04465/) (2020)
- [A Diagonal Structured State Space Model on Loihi 2 for Efficient Streaming Sequence Processing](https://mlsanity.com/paper/2409.15022/) (2024)
- [Quantizing Small-Scale State-Space Models for Edge AI](https://mlsanity.com/paper/2506.12480/) (2025)
- [IMSSA: Deploying modern state-space models on memristive in-memory compute hardware](https://mlsanity.com/paper/2412.20215/) (2024)

## Abstract (arXiv)

Structured State Space models (SSM) have recently emerged as a new class of deep learning models, particularly well-suited for processing long sequences. Their constant memory footprint, in contrast to the linearly scaling memory demands of Transformers, makes them attractive candidates for deployment on resource-constrained edge-computing devices. While recent works have explored the effect of quantization-aware training (QAT) on SSMs, they typically do not address its implications for specialized edge hardware, for example, analog in-memory computing (AIMC) chips. In this work, we demonstrate that QAT can significantly reduce the complexity of SSMs by up to two orders of magnitude across various performance metrics. We analyze the relation between model size and numerical precision, and show that QAT enhances robustness to analog noise and enables structural pruning. Finally, we integrate these techniques to deploy SSMs on a memristive analog in-memory computing substrate and highlight the resulting benefits in terms of computational efficiency.