mlsanityData to 7 Oct 2026

DCP-Prune: Ultra-Low Token Pruning with Distribution Consistency Preservation

arXiv 2606.16633 ↗First posted 15 Jun 2026cs.CV cs.AIcited by 0

On the map: Efficient LLM inference and serving → Visual token pruning in multimodal models

Abstract (arXiv)

Recent vision token pruning methods effectively preserve model performance under moderate token budgets but become unstable under ultra-low token budget. Our analysis shows that as the pruning budget decreases, accuracy degradation is often accompanied by larger feature distribution shifts. Critically, the degree of this distribution shift strongly correlates with performance degradation. To better characterize this phenomenon, we introduce a lightweight distribution consistency metric to estimate the distribution shift between retained and full tokens. Motivated by these observations, we propose a two-stage pruning framework consisting of Anchor-Context Graph Recovery (ACGR) and Text-Aware Token Cluster Selection (TATCS). Specifically, ACGR transfers contextual information before token removal, while TATCS dynamically re-selects representative tokens when severe distribution shift is detected. Extensive experiments demonstrate that our method achieves superior and more stable performance under ultra-low token budget. Notably, it retains 92.1% of the upper-bound average performance on LLaVA-1.5-7B with only 16 visual tokens.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Visual token pruning in multimodal models, one of 12 directions in Efficient LLM inference and serving. Builds on 49 of its 70 references that are in the map. First posted on arXiv 15 Jun 2026.

Builds on

49 of 70 references.

Closest ancestors · 13

Same direction, newest first

  1. 2026RCP: Representation Consistency Pruner for Mitigating Distribution Shift in Large Vision-Language Models4 Apr
  2. 2026Prune Redundancy, Preserve Essence: Vision Token Compression in VLMs via Synergistic Importance-Diversity10 Mar
  3. 2026VisionTrim: Unified Vision Token Compression for Training-Free MLLM Acceleration30 Jan
  4. 2025IPCV: Information-Preserving Compression for MLLM Visual Encoders21 Dec
  5. 2025SCOPE: Saliency-Coverage Oriented Token Pruning for Efficient Multimodel LLMs28 Oct
  6. 2025Don't Just Chase "Highlighted Tokens" in MLLMs: Revisiting Visual Holistic Context Retention3 Oct
  7. 2025VFlowOpt: A Token Pruning Framework for LMMs with Visual Information Flow-Guided Optimization7 Aug
  8. 2025Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimization28 May
  9. 2025TokenCarve: Information-Preserving Visual Token Compression in Multimodal Large Language Models13 Mar
  10. 2025LVPruning: An Effective yet Simple Language-Guided Vision Token Pruning Approach for Multi-modal Large Language Models23 Jan
  11. 2025What Kind of Visual Tokens Do We Need? Training-free Visual Token Pruning for Multi-modal Large Language Models from the Perspective of Graph4 Jan
  12. 2024Feather the Throttle: Revisiting Visual Token Pruning for Vision-Language Model Acceleration17 Dec
1 more
  1. 2024Beyond Text-Visual Attention: Exploiting Visual Cues for Effective Token Pruning in VLMs2 Dec

Other references · 36

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  6. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  7. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  8. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  9. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  10. 2019Towards VQA Models That Can Readcited by 1,696
  11. 2021ViViT: A Video Vision Transformercited by 1,610
  12. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
24 more
  1. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,194
  2. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
  3. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,175
  4. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  5. 2022Token Merging: Your ViT But Fastercited by 995
  6. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 891
  7. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 849
  8. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  9. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
  10. 2020Transformer Interpretability Beyond Attention Visualizationcited by 620
  11. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  12. 2017TGIF-QA: Toward Spatio-Temporal Reasoning in Visual Question Answeringcited by 378
  13. 2022Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizationscited by 359
  14. 2023Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolutioncited by 226
  15. 2016Correlation Alignment for Unsupervised Domain Adaptationcited by 146
  16. 2023AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Modelcited by 93
  17. 2017Towards Faster Training of Global Covariance Pooling Networks by Iterative Matrix Square Root Normalizationcited by 65
  18. 2023Zero-TPrune: Zero-Shot Token Pruning through Leveraging of the Attention Graph in Pre-Trained Transformerscited by 61
  19. 2022Beyond Attentive Tokens: Incorporating Token Importance and Diversity for Efficient Vision Transformerscited by 47
  20. 2025Similarity-Aware Token Pruning: Your VLM but Fastercited by 32
  21. 2025FlowCut: Rethinking Redundancy via Information Flow for Efficient Vision-Language Modelscited by 29
  22. 2021On the Properties of Kullback-Leibler Divergence Between Multivariate Gaussian Distributionscited by 22
  23. 2024Token Transformation Matters: Towards Faithful Post-hoc Explanation for Vision Transformercited by 19
  24. 2022Network Pruning via Feature Shift Minimizationcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025VLM-Pruner: Buffering for Spatial Sparsity in an Efficient VLM Centrifugal Token Pruning Paradigm
  2. 2026ERASE: Eliminating Redundant Visual Tokens via Adaptive Two-Stage Token Pruning
  3. 2025Why 1 + 1 < 1 in Visual Token Pruning: Beyond Naive Integration via Multi-Objective Balanced Covering
  4. 2026Adaptive Two-Stage Visual Token Pruning for Efficient Inference in Video-Language Models
  5. 2024Exploring Token Pruning in Vision State Space Models
  6. 2026Centering before Pruning: Lightweight Geometry Correction for Diversity-Based Visual Token Pruning in LVLMs
  7. 2025TrimTokenator-LC: Towards Adaptive Visual Token Pruning for Large Multimodal Models with Long Contexts
  8. 2025When Token Pruning is Worse than Random: Understanding Visual Token Information in VLLMs