mlsanityData to 14 Oct 2026

QuarterMap: Efficient Post-Training Token Pruning for Visual State Space Models

arXiv 2507.09514 ↗First posted 13 Jul 2025cs.CV cs.AIcited by 0

On the map: Efficient LLM inference and serving → Token pruning and merging for ViTs

Abstract (arXiv)

State space models (SSMs) reduce the quadratic complexity of transformers by leveraging linear recurrence. Recently, VMamba has emerged as a strong SSM-based vision backbone, yet remains bottlenecked by spatial redundancy in its four-directional scan. We propose QuarterMap, a post-training activation pruning method that removes redundant spatial activations before scanning and restores dimensions via nearest-neighbor upsampling. Our method improves throughput without retraining. On ImageNet-1K, QuarterMap achieves up to 11% speedup on VMamba with less than 0.9% accuracy drop, and yields similar gains on ADE20K segmentation. Beyond VMamba, we validate QuarterMap on MedMamba, a domain-specific model that shares the same four-directional scanning structure, where it consistently improves throughput while preserving accuracy across multiple medical imaging tasks. Compared to token merging methods like ToMe, QuarterMap is tailored for SSMs and avoids costly merge-unmerge operations. Our method offers a plug-and-play tool for deployment-time efficiency without compromising transferability.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Token pruning and merging for ViTs, one of 12 directions in Efficient LLM inference and serving. Builds on 48 of its 61 references that are in the map. First posted on arXiv 13 Jul 2025.

Builds on

48 of 61 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Faster Vision Mamba is Rebuilt in Minutes via Merged Token Re-training17 Dec
  2. 2024Exploring Token Pruning in Vision State Space Models27 Sep
  3. 2022Token Merging: Your ViT But Faster17 Oct

Other references · 45

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  6. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  7. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  8. 2020Longformer: The Long-Document Transformercited by 3,413
  9. 2015Fast R-CNNcited by 2,933
  10. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  11. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  12. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
33 more
  1. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  2. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
  3. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,271
  4. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,228
  5. 2016Pruning Filters for Efficient ConvNetscited by 1,143
  6. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,056
  7. 2024VMamba: Visual State Space Modelcited by 969
  8. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  9. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 827
  10. 2022Simplified State Space Layers for Sequence Modelingcited by 788
  11. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 702
  12. 2021MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classificationcited by 690
  13. 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 605
  14. 2022Hungry Hungry Hippos: Towards Language Modeling with State Space Modelscited by 499
  15. 2021AdaViT: Adaptive Tokens for Efficient Vision Transformercited by 362
  16. 2024VideoMamba: State Space Model for Efficient Video Understandingcited by 328
  17. 2018Soft Filter Pruning for Accelerating Deep Convolutional Neural Networkscited by 315
  18. 2020HRank: Filter Pruning using High-Rank Feature Mapcited by 292
  19. 2021Post-Training Quantization for Vision Transformercited by 278
  20. 2024SegMamba: Long-range Sequential Modeling Mamba For 3D Medical Image Segmentationcited by 266
  21. 2017NISP: Pruning Networks using Neuron Importance Score Propagationcited by 228
  22. 2023Token Merging for Fast Stable Diffusioncited by 224
  23. 2023EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attentioncited by 172
  24. 2018Discrimination-aware Channel Pruning for Deep Neural Networkscited by 171
  25. 2021FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformercited by 151
  26. 2024EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mambacited by 146
  27. 2024PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognitioncited by 141
  28. 2021Patch Slimming for Efficient Vision Transformerscited by 133
  29. 2024Mamba-UNet: UNet-Like Pure Visual Mamba for Medical Image Segmentationcited by 133
  30. 2024DiM: Diffusion Mamba for Efficient High-Resolution Image Synthesiscited by 72
  31. 2024VMRNN: Integrating Vision Mamba and LSTM for Efficient and Accurate Spatiotemporal Forecastingcited by 35
  32. 2022Dynamic Spatial Sparsification for Efficient Vision Transformers and Convolutional Neural Networkscited by 24
  33. 2023Supervised Masked Knowledge Distillation for Few-Shot Transformerscited by 23

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024V"Mean"ba: Visual State Space Models only need 1 hidden dimension
  2. 2024QuadMamba: Learning Quadtree-based Selective Scan for Visual State Space Model
  3. 2023Training-Free Acceleration of ViTs with Delayed Spatial Merging
  4. 2026StructSAM: Structure- and Spectrum-Preserving Token Merging for Segment Anything Models
  5. 2026MFil-Mamba: Multi-Filter Scanning for Spatial Redundancy-Aware Visual State Space Models
  6. 2023PPT: Token Pruning and Pooling for Efficient Vision Transformers
  7. 2025VLM-Pruner: Buffering for Spatial Sparsity in an Efficient VLM Centrifugal Token Pruning Paradigm
  8. 2025Similarity-Aware Token Pruning: Your VLM but Faster