# Quamba: A Post-Training Quantization Recipe for Selective State Space Models

arXiv: https://arxiv.org/abs/2410.13229 · HTML: https://mlsanity.com/paper/2410.13229/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 48 of its 61 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Placed in

- State-space sequence models (S4, Mamba) / [Core SSM architectures for language](https://mlsanity.com/ssm/core-ssm-architectures-for-language/)

## Builds on (most cited first, 30 of 48)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Root Mean Square Layer Normalization](https://mlsanity.com/paper/1910.07467/) (2019)
- [SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models](https://mlsanity.com/paper/2211.10438/) (2022)
- [Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model](https://mlsanity.com/paper/2401.09417/) (2024)
- [The LAMBADA dataset: Word prediction requiring a broad discourse context](https://mlsanity.com/paper/1606.06031/) (2016)
- [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1702.03118/) (2017)
- [RWKV: Reinventing RNNs for the Transformer Era](https://mlsanity.com/paper/2305.13048/) (2023)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2208.04933/) (2022)
- [QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs](https://mlsanity.com/paper/2404.00456/) (2024)
- [HiPPO: Recurrent Memory with Optimal Polynomial Projections](https://mlsanity.com/paper/2008.07669/) (2020)
- [Zephyr: Direct Distillation of LM Alignment](https://mlsanity.com/paper/2310.16944/) (2023)
- [HAQ: Hardware-Aware Automated Quantization with Mixed Precision](https://mlsanity.com/paper/1811.08886/) (2018)
- [On the Parameterization and Initialization of Diagonal State Space Models](https://mlsanity.com/paper/2206.11893/) (2022)
- [Jamba: A Hybrid Transformer-Mamba Language Model](https://mlsanity.com/paper/2403.19887/) (2024)
- [SliceGPT: Compress Large Language Models by Deleting Rows and Columns](https://mlsanity.com/paper/2401.15024/) (2024)
- [QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks](https://mlsanity.com/paper/2402.04396/) (2024)

## Built on by (most cited first, 28 of 28)

- [Quamba2: A Robust and Scalable Post-training Quantization Framework for Selective State Space Models](https://mlsanity.com/paper/2503.22879/) (2025)
- [Neuromorphic Principles for Efficient Large Language Models on Intel Loihi 2](https://mlsanity.com/paper/2503.18002/) (2025)
- [UniQL: Unified Quantization and Low-rank Compression for Adaptive Edge LLMs](https://mlsanity.com/paper/2512.03383/) (2025)
- [On State Reduction in Linear Attention](https://mlsanity.com/paper/2602.04852/) (2026)
- [DAMP: Decay-Aware Mixed-Precision Recurrent-State Quantization](https://mlsanity.com/paper/2608.27513/) (2026)
- [OuroMamba: A Data-Free Quantization Framework for Vision Mamba](https://mlsanity.com/paper/2503.10959/) (2025)
- [Q-MambaIR: Accurate Quantized Mamba for Efficient Image Restoration](https://mlsanity.com/paper/2503.21970/) (2025)
- [Post-Training Quantization for Vision Mamba with k-Scaled Quantization and Reparameterization](https://mlsanity.com/paper/2501.16738/) (2025)
- [The Curious Case of In-Training Compression of State Space Models](https://mlsanity.com/paper/2510.02823/) (2025)
- [Quantizing Small-Scale State-Space Models for Edge AI](https://mlsanity.com/paper/2506.12480/) (2025)
- [SSDi8: Accurate and Efficient 8-bit Quantization for State Space Duality](https://mlsanity.com/paper/2608.21952/) (2026)
- [Low-Bit Data Processing Using Multiple-Output Spiking Neurons with Non-linear Reset Feedback](https://mlsanity.com/paper/2508.06292/) (2025)
- [Ternary Mamba: Grouped Quantization-Aware Training of W1.58A16 State Space Models](https://mlsanity.com/paper/2606.18114/) (2026)
- [Quantizing Looped Transformers: Feedback Exposure and Calibration Blindness](https://mlsanity.com/paper/2609.30820/) (2026)
- [QS4D: Quantization-aware training for efficient hardware deployment of structured state-space sequential models](https://mlsanity.com/paper/2507.06079/) (2025)
- [Sequential Token Merging: Revisiting Hidden States](https://mlsanity.com/paper/2509.22691/) (2025)
- [Late Breaking Results: Quamba-SE: Soft-edge Quantizer for Activations in State Space Models](https://mlsanity.com/paper/2601.09451/) (2026)
- [COREY: Entropy-Guided Runtime Chunk Scheduling for Selective Scan Kernels](https://mlsanity.com/paper/2604.10597/) (2026)
- [ViM-Q: Scalable Algorithm-Hardware Co-Design for Vision Mamba Model Inference on FPGA](https://mlsanity.com/paper/2605.01935/) (2026)
- [A Circuit, Not The Circuit: Non-Unique Causal Localisation of the Mamba-2 State Sink](https://mlsanity.com/paper/2606.00930/) (2026)
- [Energy- and Memory-Efficient PEFT Methods for Personalized On-Device SLMs on Consumer GPUs](https://mlsanity.com/paper/2608.04488/) (2026)
- [Channel-Wise and Token-Aware Post-Training Quantization for Visual State Space Duality](https://mlsanity.com/paper/2609.16656/) (2026)
- [LeapQuant: Efficient Linear Attention with Accurate Recurrent State Quantization](https://mlsanity.com/paper/2609.38166/) (2026)
- [STEPQuant: When and Where Errors Matter in Delta-Rule Recurrent State Quantization](https://mlsanity.com/paper/2609.38169/) (2026)
- [Low-Discrepancy Dither for Quantized Recurrent State Caches](https://mlsanity.com/paper/2609.39185/) (2026)
- [Saying, Not Knowing: Aggressively GGUF-Quantized Small Language Models Still Write Rare Words They Can No Longer Define](https://mlsanity.com/paper/2610.04403/) (2026)
- [Preserving Updates in Low-Bit Recurrent State Quantization](https://mlsanity.com/paper/2610.15709/) (2026)
- [Hardware-Friendly Token-Group Activation Quantization for Low-Bit Mamba Super-Resolution](https://mlsanity.com/paper/2610.16261/) (2026)

## Abstract (arXiv)

State Space Models (SSMs) have emerged as an appealing alternative to Transformers for large language models, achieving state-of-the-art accuracy with constant memory complexity which allows for holding longer context lengths than attention-based networks. The superior computational efficiency of SSMs in long sequence modeling positions them favorably over Transformers in many scenarios. However, improving the efficiency of SSMs on request-intensive cloud-serving and resource-limited edge applications is still a formidable task. SSM quantization is a possible solution to this problem, making SSMs more suitable for wide deployment, while still maintaining their accuracy. Quantization is a common technique to reduce the model size and to utilize the low bit-width acceleration features on modern computing units, yet existing quantization techniques are poorly suited for SSMs. Most notably, SSMs have highly sensitive feature maps within the selective scan mechanism (i.e., linear recurrence) and massive outliers in the output activations which are not present in the output of token-mixing in the self-attention modules. To address this issue, we propose a static 8-bit per-tensor SSM quantization method which suppresses the maximum values of the input activations to the selective SSM for finer quantization precision and quantizes the output activations in an outlier-free space with Hadamard transform. Our 8-bit weight-activation quantized Mamba 2.8B SSM benefits from hardware acceleration and achieves a 1.72x lower generation latency on an Nvidia Orin Nano 8G, with only a 0.9% drop in average accuracy on zero-shot tasks. The experiments demonstrate the effectiveness and practical applicability of our approach for deploying SSM-based models of all sizes on both cloud and edge platforms.