mlsanityData to 7 Oct 2026

Post-Training Quantization for Vision Mamba with k-Scaled Quantization and Reparameterization

arXiv 2501.16738 ↗First posted 28 Jan 2025eess.IVcited by 4

On the map: State-space sequence models (S4, Mamba) → Vision Mamba backbones

Abstract (arXiv)

The Mamba model, utilizing a structured state-space model (SSM), offers linear time complexity and demonstrates significant potential. Vision Mamba (ViM) extends this framework to vision tasks by incorporating a bidirectional SSM and patch embedding, surpassing Transformer-based models in performance. While model quantization is essential for efficient computing, existing works have focused solely on the original Mamba model and have not been applied to ViM. Additionally, they neglect quantizing the SSM layer, which is central to Mamba and can lead to substantial error propagation by naive quantization due to its inherent structure. In this paper, we focus on the post-training quantization (PTQ) of ViM. We address the issues with three core techniques: 1) a k-scaled token-wise quantization method for linear and convolutional layers, 2) a reparameterization technique to simplify hidden state quantization, and 3) a factor-determining method that reduces computational overhead by integrating operations. Through these methods, the error caused by PTQ can be mitigated. Experimental results on ImageNet-1k demonstrate only a 0.8-1.2% accuracy degradation due to PTQ, highlighting the effectiveness of our approach.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Vision Mamba backbones, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 22 of its 23 references that are in the map. First posted on arXiv 28 Jan 2025.

Builds on

22 of 23 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024MambaVision: A Hybrid Mamba-Transformer Vision Backbone10 Jul
  2. 2024EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba15 Mar
  3. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model17 Jan

Other references · 19

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  3. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  4. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  5. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  6. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,283
  7. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  8. 2024VMamba: Visual State Space Modelcited by 955
  9. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  10. 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 185
  11. 2020Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluationcited by 183
  12. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 173
7 more
  1. 2021FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformercited by 148
  2. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 134
  3. 2020EasyQuant: Post-training Quantization via Scale Optimizationcited by 57
  4. 2024Quamba: A Post-Training Quantization Recipe for Selective State Space Modelscited by 25
  5. 2024Mamba-PTQ: Outlier Channels in Recurrent Large Language Modelscited by 15
  6. 2024MPTQ-ViT: Mixed-Precision Post-Training Quantization for Vision Transformercited by 10
  7. 2023TSPTQ-ViT: Two-scaled post-training quantization for vision transformercited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025OuroMamba: A Data-Free Quantization Framework for Vision Mambacited by 5
  2. 2025Sequential Token Merging: Revisiting Hidden Statescited by 0
  3. 2026ViM-Q: Scalable Algorithm-Hardware Co-Design for Vision Mamba Model Inference on FPGAcited by 0
  4. 2026Channel-Wise and Token-Aware Post-Training Quantization for Visual State Space Dualitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024PTQ4VM: Post-Training Quantization for Visual Mamba
  2. 2025QMamba: Post-Training Quantization for Vision State Space Models
  3. 2025ViM-VQ: Efficient Post-Training Vector Quantization for Visual Mamba
  4. 2026Beyond ZOH: Advanced Discretization Strategies for Vision Mamba
  5. 2025Q-MambaIR: Accurate Quantized Mamba for Efficient Image Restoration
  6. 2025MambaQuant: Quantizing the Mamba Family with Variance Aligned Rotation Methods
  7. 2024MambaOut: Do We Really Need Mamba for Vision?
  8. 2025Fast Vision Mamba: Pooling Spatial Dimensions for Accelerated Processing