mlsanityData to 14 Oct 2026

Mamba-PTQ: Outlier Channels in Recurrent Large Language Models

arXiv 2407.12397 ↗First posted 17 Jul 2024cs.LG cs.AI cs.NEcited by 15

On the map: State-space sequence models (S4, Mamba) → Core SSM architectures for language

Abstract (arXiv)

Modern recurrent layers are emerging as a promising path toward edge deployment of foundation models, especially in the context of large language models (LLMs). Compressing the whole input sequence in a finite-dimensional representation enables recurrent layers to model long-range dependencies while maintaining a constant inference cost for each token and a fixed memory requirement. However, the practical deployment of LLMs in resource-limited environments often requires further model compression, such as quantization and pruning. While these techniques are well-established for attention-based models, their effects on recurrent layers remain underexplored. In this preliminary work, we focus on post-training quantization for recurrent LLMs and show that Mamba models exhibit the same pattern of outlier channels observed in attention-based LLMs. We show that the reason for the difficulty of quantizing SSMs is caused by activation outliers, similar to those observed in transformer-based LLMs. We report baseline results for post-training quantization of Mamba that do not take into account the activation outliers and suggest first steps for outlier-aware quantization.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 20 of its 27 references that are in the map. First posted on arXiv 17 Jul 2024.

Builds on

20 of 27 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Jamba: A Hybrid Transformer-Mamba Language Model28 Mar
  2. 2021Efficiently Modeling Long Sequences with Structured State Spaces31 Oct

Other references · 18

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  5. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  6. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  7. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  8. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 992
  9. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  10. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 827
  11. 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 605
  12. 2024Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Modelscited by 242
6 more
  1. 2023Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothingcited by 151
  2. 2021Do Long-Range Language Models Actually Use Long-Range Context?cited by 92
  3. 2024RecurrentGemma: Moving Past Transformers for Efficient Open Language Modelscited by 64
  4. 2023Efficient Video and Audio processing with Loihi 2cited by 29
  5. 2020Hardware Aware Training for Efficient Keyword Spotting on General Purpose and Specialized Hardwarecited by 8
  6. 2024FlattenQuant: Breaking Through the Inference Compute-bound for Large Language Models with Per-tensor Quantizationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 7; 2026: 6.

15 papers.

Sort:

By area:

  1. 2024Quamba: A Post-Training Quantization Recipe for Selective State Space Modelscited by 28
  2. 2024Bi-Mamba: Towards Accurate 1-Bit State Space Modelscited by 12
  3. 2025LightMamba: Efficient Mamba Acceleration on FPGA with Quantization and Hardware Co-designcited by 11
  4. 2025Neuromorphic Principles for Efficient Large Language Models on Intel Loihi 2cited by 8
  5. 2025OuroMamba: A Data-Free Quantization Framework for Vision Mambacited by 5
  6. 2025Post-Training Quantization for Vision Mamba with k-Scaled Quantization and Reparameterizationcited by 4
  7. 2025Accelerating Linear Recurrent Neural Networks for the Edge with Unstructured Sparsitycited by 3
  8. 2025Quantizing Small-Scale State-Space Models for Edge AIcited by 3
  9. 2026A KL Lens on Quantization: Fast, Forward-Only Sensitivity for Mixed-Precision SSM-Transformer Modelscited by 2
  10. 2025Neural Weight Compression for Language Modelscited by 1
  11. 2026Decoder-Free Distillation for Quantized Image Restorationcited by 1
  12. 2026Ternary Mamba: Grouped Quantization-Aware Training of W1.58A16 State Space Modelscited by 1
3 more
  1. 2026TinyCast: Probabilistic Zero-Shot Forecasting with Computed Periodicitycited by 1
  2. 2026Late Breaking Results: Quamba-SE: Soft-edge Quantizer for Activations in State Space Modelscited by 0
  3. 2026ViM-Q: Scalable Algorithm-Hardware Co-Design for Vision Mamba Model Inference on FPGAcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024OutlierTune: Efficient Channel-Wise Quantization for Large Language Models
  2. 2023QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models
  3. 2025Rethinking the Outlier Distribution in Large Language Models: An In-depth Study
  4. 2023Rethinking Channel Dimensions to Isolate Outliers for Low-bit Weight Quantization of Large Language Models
  5. 2023Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling
  6. 2024Outlier Reduction with Gated Attention for Improved Post-training Quantization in Large Sequence-to-sequence Speech Foundation Models
  7. 2024Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization
  8. 2025How Can Mamba Learn In Context with Outliers and Generalize Provably?