mlsanityData to 14 Oct 2026

Reg4Pru: Regularisation Through Random Token Routing for Token Pruning

arXiv 2602.02163 ↗First posted 2 Feb 2026cs.CVcited by 0

On the map: Efficient LLM inference and serving → Token pruning and merging for ViTs

Abstract (arXiv)

Transformers are widely adopted in modern vision models due to their strong ability to scale with dataset size and generalisability. However, this comes with a major drawback: computation scales quadratically to the total number of tokens. Numerous methods have been proposed to mitigate this. For example, we consider token pruning with reactivating tokens from preserved representations, but the increased computational efficiency of this method results in decreased stability from the preserved representations, leading to poorer dense prediction performance at deeper layers. In this work, we introduce Reg4Pru, a training regularisation technique that mitigates token-pruning performance loss for segmentation. We compare our models on the FIVES blood vessel segmentation dataset and find that Reg4Pru improves average precision by an absolute 46% compared to the same model trained without routing. This increase is observed using a configuration that achieves a 29% relative speedup in wall-clock time compared to the non-pruned baseline. These findings indicate that Reg4Pru is a valuable regulariser for token reduction strategies.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Token pruning and merging for ViTs, one of 12 directions in Efficient LLM inference and serving. Builds on 35 of its 44 references that are in the map. First posted on arXiv 2 Feb 2026.

Builds on

35 of 44 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Dynamic Token Pruning in Plain Vision Transformers for Semantic Segmentation2 Aug
  2. 2023Revisiting Token Pruning for Object Detection and Instance Segmentation12 Jun
  3. 2023Token Sparsification for Faster Medical Image Segmentation11 Mar
  4. 2022Token Merging: Your ViT But Faster17 Oct
  5. 2021SPViT: Enabling Faster Vision Transformers via Soft Token Pruning27 Dec

Other references · 30

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  5. 2020End-to-End Object Detection with Transformerscited by 7,414
  6. 2016Layer Normalizationcited by 5,669
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  8. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  9. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  10. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,351
  11. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  12. 2020Reformer: The Efficient Transformercited by 1,668
18 more
  1. 2019Root Mean Square Layer Normalizationcited by 1,480
  2. 2016Deep Networks with Stochastic Depthcited by 979
  3. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  4. 2021Going deeper with Image Transformerscited by 764
  5. 2022Vision Transformer Adapter for Dense Predictionscited by 565
  6. 2018Albumentations: fast and flexible image augmentationscited by 560
  7. 2025Perception Encoder: The best visual embeddings are not at the output of the networkcited by 326
  8. 2024The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bitscited by 308
  9. 2024Mixture-of-Depths: Dynamically allocating compute in transformer-based language modelscited by 258
  10. 2024Rotary Position Embedding for Vision Transformercited by 236
  11. 2023Token Merging for Fast Stable Diffusioncited by 224
  12. 2024Flex Attention: A Programming Model for Generating Optimized Attention Kernelscited by 185
  13. 2024What Matters in Transformers? Not All Attention is Neededcited by 84
  14. 2025Your ViT is Secretly an Image Segmentation Modelcited by 63
  15. 2025Scaling Vision Pre-Training to 4K Resolutioncited by 28
  16. 2025TREAD: Token Routing for Efficient Architecture-agnostic Diffusion Trainingcited by 21
  17. 2022Neural Architecture Search on Efficient Transformers and Beyondcited by 17
  18. 2021Layer-wise Pruning of Transformer Attention Heads for Efficient Language Modelingcited by 14

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Where Do Tokens Go? Understanding Pruning Behaviors in STEP at High Resolutions
  2. 2025Prompt-based Dynamic Token Pruning for Efficient Segmentation of Medical Images
  3. 2025Token Transforming: A Unified and Training-Free Token Compression Framework for Vision Transformer Acceleration
  4. 2025Similarity-Aware Token Pruning: Your VLM but Faster
  5. 2023PPT: Token Pruning and Pooling for Efficient Vision Transformers
  6. 2024Token Cropr: Faster ViTs for Quite a Few Tasks
  7. 2026DCP-Prune: Ultra-Low Token Pruning with Distribution Consistency Preservation
  8. 2026Scaling Vision Transformers Without Training: Prune-and-Merge Token Compression