# PPT: Token Pruning and Pooling for Efficient Vision Transformers

arXiv: https://arxiv.org/abs/2310.01812 · HTML: https://mlsanity.com/paper/2310.01812/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Placed in Token pruning and merging for ViTs, one of 12 directions in Efficient LLM inference and serving.
- Builds on 31 of its 38 references that are in the map.
- First posted on arXiv 3 Oct 2023.

## Placed in

- Efficient LLM inference and serving / [Token pruning and merging for ViTs](https://mlsanity.com/inference/token-pruning-and-merging-for-vits/)

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [Token Merging: Your ViT But Faster](https://mlsanity.com/paper/2210.09461/) (2022)
- [Designing Network Design Spaces](https://mlsanity.com/paper/2003.13678/) (2020)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification](https://mlsanity.com/paper/2106.02034/) (2021)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [Exploring Plain Vision Transformer Backbones for Object Detection](https://mlsanity.com/paper/2203.16527/) (2022)
- [Transformer in Transformer](https://mlsanity.com/paper/2103.00112/) (2021)
- [Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizations](https://mlsanity.com/paper/2202.07800/) (2022)
- [AdaViT: Adaptive Tokens for Efficient Vision Transformer](https://mlsanity.com/paper/2112.07658/) (2021)
- [Rethinking Spatial Dimensions of Vision Transformers](https://mlsanity.com/paper/2103.16302/) (2021)
- [Co-Scale Conv-Attentional Image Transformers](https://mlsanity.com/paper/2104.06399/) (2021)
- [Adaptive Token Sampling For Efficient Vision Transformers](https://mlsanity.com/paper/2111.15667/) (2021)
- [Evo-ViT: Slow-Fast Token Evolution for Dynamic Vision Transformer](https://mlsanity.com/paper/2108.01390/) (2021)
- [All Tokens Matter: Token Labeling for Training Better Vision Transformers](https://mlsanity.com/paper/2104.10858/) (2021)
- [Patch Slimming for Efficient Vision Transformers](https://mlsanity.com/paper/2106.02852/) (2021)
- [IA-RED2: Interpretability-Aware Redundancy Reduction for Vision Transformers](https://mlsanity.com/paper/2106.12620/) (2021)
- [Joint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformers](https://mlsanity.com/paper/2304.10716/) (2023)
- [Token Pooling in Vision Transformers](https://mlsanity.com/paper/2110.03860/) (2021)

## Built on by (most cited first, 30 of 38)

- [DiTFastAttn: Attention Compression for Diffusion Transformer Models](https://mlsanity.com/paper/2406.08552/) (2024)
- [Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodality](https://mlsanity.com/paper/2505.18227/) (2025)
- [Similarity-Aware Token Pruning: Your VLM but Faster](https://mlsanity.com/paper/2503.11549/) (2025)
- [D2O: Dynamic Discriminative Operations for Efficient Long-Context Inference of Large Language Models](https://mlsanity.com/paper/2406.13035/) (2024)
- [ALGM: Adaptive Local-then-Global Token Merging for Efficient Semantic Segmentation with Plain Vision Transformers](https://mlsanity.com/paper/2406.09936/) (2024)
- [Importance-Based Token Merging for Efficient Image and Video Generation](https://mlsanity.com/paper/2411.16720/) (2024)
- [ST3: Accelerating Multimodal Large Language Model by Spatial-Temporal Visual Token Trimming](https://mlsanity.com/paper/2412.20105/) (2024)
- [MambaIC: State Space Models for High-Performance Learned Image Compression](https://mlsanity.com/paper/2503.12461/) (2025)
- [Token Cropr: Faster ViTs for Quite a Few Tasks](https://mlsanity.com/paper/2412.00965/) (2024)
- [SLAB: Efficient Transformers with Simplified Linear Attention and Progressive Re-parameterized Batch Normalization](https://mlsanity.com/paper/2405.11582/) (2024)
- [LLMCBench: Benchmarking Large Language Model Compression for Efficient Deployment](https://mlsanity.com/paper/2410.21352/) (2024)
- [Turbo: Informativity-Driven Acceleration Plug-In for Vision-Language Large Models](https://mlsanity.com/paper/2407.11717/) (2024)
- [Agglomerative Token Clustering](https://mlsanity.com/paper/2409.11923/) (2024)
- [Grouping First, Attending Smartly: Training-Free Acceleration for Diffusion Transformers](https://mlsanity.com/paper/2505.14687/) (2025)
- [Accelerating Vision Transformers with Adaptive Patch Sizes](https://mlsanity.com/paper/2510.18091/) (2025)
- [FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality Samples with Less Compute](https://mlsanity.com/paper/2502.20126/) (2025)
- [Continual Learning for Generative AI: From LLMs to MLLMs and Beyond](https://mlsanity.com/paper/2506.13045/) (2025)
- [ToSA: Token Merging with Spatial Awareness](https://mlsanity.com/paper/2506.20066/) (2025)
- [DyMU: Dynamic Merging and Virtual Unmerging for Efficient VLMs](https://mlsanity.com/paper/2504.17040/) (2025)
- [Token Transforming: A Unified and Training-Free Token Compression Framework for Vision Transformer Acceleration](https://mlsanity.com/paper/2506.05709/) (2025)
- [Fine-Grained Post-Training Quantization for Large Vision Language Models with Quantization-Aware Integrated Gradients](https://mlsanity.com/paper/2603.17809/) (2026)
- [You Only Need Less Attention at Each Stage in Vision Transformers](https://mlsanity.com/paper/2406.00427/) (2024)
- [ExCP: Extreme LLM Checkpoint Compression via Weight-Momentum Joint Shrinking](https://mlsanity.com/paper/2406.11257/) (2024)
- [Token Compensator: Altering Inference Cost of Vision Transformer without Re-Tuning](https://mlsanity.com/paper/2408.06798/) (2024)
- [TinyChemVL: Advancing Chemical Vision-Language Models via Efficient Visual Token Reduction and Complex Reaction Tasks](https://mlsanity.com/paper/2511.06283/) (2025)
- [Efficient Sparse Attention needs Adaptive Token Release](https://mlsanity.com/paper/2407.02328/) (2024)
- [Purge-Gate: Backpropagation-Free Test-Time Adaptation for Point Clouds Classification via Token Purging](https://mlsanity.com/paper/2509.09785/) (2025)
- [GreedyPrune: Retenting Critical Visual Token Set for Large Vision Language Models](https://mlsanity.com/paper/2506.13166/) (2025)
- [Collaborative Multi-Mode Pruning for Vision-Language Models](https://mlsanity.com/paper/2604.02956/) (2026)
- [MPM: Mutual Pair Merging for Efficient Vision Transformers](https://mlsanity.com/paper/2604.05718/) (2026)

## Abstract (arXiv)

Vision Transformers (ViTs) have emerged as powerful models in the field of computer vision, delivering superior performance across various vision tasks. However, the high computational complexity poses a significant barrier to their practical applications in real-world scenarios. Motivated by the fact that not all tokens contribute equally to the final predictions and fewer tokens bring less computational cost, reducing redundant tokens has become a prevailing paradigm for accelerating vision transformers. However, we argue that it is not optimal to either only reduce inattentive redundancy by token pruning, or only reduce duplicative redundancy by token merging. To this end, in this paper we propose a novel acceleration framework, namely token Pruning & Pooling Transformers (PPT), to adaptively tackle these two types of redundancy in different layers. By heuristically integrating both token pruning and token pooling techniques in ViTs without additional trainable parameters, PPT effectively reduces the model complexity while maintaining its predictive accuracy. For example, PPT reduces over 37% FLOPs and improves the throughput by over 45% for DeiT-S without any accuracy drop on the ImageNet dataset. The code is available at https://github.com/xjwu1024/PPT and https://github.com/mindspore-lab/models/