Back to Fundamentals: Low-Level Visual Features Guided Progressive Token Pruning
On the map: Efficient LLM inference and serving → Token pruning and merging for ViTs
Abstract (arXiv)
Vision Transformers (ViTs) excel in semantic segmentation but demand significant computation, posing challenges for deployment on resource-constrained devices. Existing token pruning methods often overlook fundamental visual data characteristics. This study introduces 'LVTP', a progressive token pruning framework guided by multi-scale Tsallis entropy and low-level visual features with twice clustering. It integrates high-level semantics and basic visual attributes for precise segmentation. A novel dynamic scoring mechanism using multi-scale Tsallis entropy weighting overcomes limitations of traditional single-parameter entropy. The framework also incorporates low-level feature analysis to preserve critical edge information while optimizing computational cost. As a plug-and-play module, it requires no architectural changes or additional training. Evaluations across multiple datasets show 20%-45% computational reductions with negligible performance loss, outperforming existing methods in balancing cost and accuracy, especially in complex edge regions.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Token pruning and merging for ViTs, one of 12 directions in Efficient LLM inference and serving. Builds on 45 of its 61 references that are in the map. First posted on arXiv 25 Apr 2025.
Builds on
45 of 61 references.
Closest ancestors · 7
Same direction, newest first
- 2024VLTP: Vision-Language Guided Token Pruning for Task-Oriented Segmentation13 Sep
- 2023Dynamic Token Pruning in Plain Vision Transformers for Semantic Segmentation2 Aug
- 2022HeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformers15 Nov
- 2022Token Merging: Your ViT But Faster17 Oct
- 2022SaiT: Sparse Vision Transformers through Adaptive Token Pruning11 Oct
- 2021SPViT: Enabling Faster Vision Transformers via Soft Token Pruning27 Dec
- 2021Learned Token Pruning for Transformers2 Jul
Other references · 38
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2023Segment Anythingcited by 8,234
- 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
- 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
- 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,558
- 2021Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentationcited by 1,222
- 2021Transformers in Vision: A Surveycited by 1,013
- 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
- 2022Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Imagescited by 830
- 2019Are Sixteen Heads Really Better than One?cited by 816
26 more
- 2020A Survey on Visual Transformercited by 654
- 2019Stand-Alone Self-Attention in Vision Modelscited by 542
- 2019YOLACT: Real-time Instance Segmentationcited by 443
- 2019Reducing Transformer Depth on Demand with Structured Dropoutcited by 439
- 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 402
- 2022Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizationscited by 363
- 2021AdaViT: Adaptive Tokens for Efficient Vision Transformercited by 362
- 2019Q-BERT: Hessian Based Ultra Low Precision Quantization of BERTcited by 359
- 2015LSTM: A Search Space Odysseycited by 316
- 2019Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challengescited by 314
- 2021Post-Training Quantization for Vision Transformercited by 278
- 2021Adaptive Token Sampling For Efficient Vision Transformerscited by 183
- 2022NeW CRFs: Neural Window Fully-connected CRFs for Monocular Depth Estimationcited by 153
- 2021FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformercited by 151
- 2021TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?cited by 127
- 2021IA-RED2: Interpretability-Aware Redundancy Reduction for Vision Transformerscited by 127
- 2021A Survey of Visual Transformerscited by 117
- 2020A Survey on Instance Segmentation: State of the artcited by 85
- 2023Towards Accurate Post-Training Quantization for Vision Transformercited by 71
- 2023A survey on efficient vision transformers: algorithms, techniques, and performance benchmarkingcited by 41
- 2023SegViTv2: Exploring Efficient and Continual Semantic Segmentation with Plain Vision Transformerscited by 25
- 2019What Object Should I Use? - Task Driven Object Detectioncited by 22
- 2023Content-aware Token Sharing for Efficient Semantic Segmentation with Vision Transformerscited by 22
- 2023RIO: A Benchmark for Reasoning Intention-Oriented Objects in Open Environmentscited by 15
- 2024Comprehensive Survey of Model Compression and Speed up for Vision Transformerscited by 14
- 2024Semantic Layering in Room Segmentation via LLMscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Where Do Tokens Go? Understanding Pruning Behaviors in STEP at High Resolutions
- 2023Multi-Scale And Token Mergence: Make Your ViT More Efficient
- 2026When Token Compression Breaks: Structural Pruning vs. Token Reduction for Robust ViT Segmentation under High Compression
- 2023Revisiting Token Pruning for Object Detection and Instance Segmentation
- 2026RAPID: Layer-Wise Redundancy-Aware Pruning and Importance-Driven Token Merging for Efficient ViT
- 2024Token Cropr: Faster ViTs for Quite a Few Tasks
- 2025Similarity-Aware Token Pruning: Your VLM but Faster
- 2023Token Fusion: Bridging the Gap between Token Pruning and Token Merging