mlsanityData to 14 Oct 2026

Neighbor-Aware Token Reduction via Hilbert Curve for Vision Transformers

arXiv 2512.22760 ↗First posted 28 Dec 2025cs.CVcited by 0

On the map: Efficient LLM inference and serving → Token pruning and merging for ViTs

Abstract (arXiv)

Vision Transformers (ViTs) have achieved remarkable success in visual recognition tasks, but redundant token representations limit their computational efficiency. Existing token merging and pruning strategies often overlook spatial continuity and neighbor relationships, resulting in the loss of local context. This paper proposes novel neighbor-aware token reduction methods based on Hilbert curve reordering, which explicitly preserves the neighbor structure in a 2D space using 1D sequential representations. Our method introduces two key strategies: Neighbor-Aware Pruning (NAP) for selective token retention and Merging by Adjacent Token similarity (MAT) for local token aggregation. Experiments demonstrate that our approach achieves state-of-the-art accuracy-efficiency trade-offs compared to existing methods. This work highlights the importance of spatial continuity and neighbor structure, offering new insights for the architectural optimization of ViTs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Token pruning and merging for ViTs, one of 12 directions in Efficient LLM inference and serving. Builds on 27 of its 34 references that are in the map. First posted on arXiv 28 Dec 2025.

Builds on

27 of 34 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024Learning to Merge Tokens via Decoupled Embedding for Efficient Vision Transformers13 Dec
  2. 2023GTP-ViT: Efficient Vision Transformers via Graph-based Token Propagation6 Nov
  3. 2023DiffRate : Differentiable Compression Rate for Efficient Vision Transformers29 May
  4. 2023Zero-TPrune: Zero-Shot Token Pruning through Leveraging of the Attention Graph in Pre-Trained Transformers27 May
  5. 2023Joint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformers21 Apr
  6. 2022Token Merging: Your ViT But Faster17 Oct
  7. 2022SaiT: Sparse Vision Transformers through Adaptive Token Pruning11 Oct
  8. 2021SPViT: Enabling Faster Vision Transformers via Soft Token Pruning27 Dec
  9. 2021Evo-ViT: Slow-Fast Token Evolution for Dynamic Vision Transformer3 Aug

Other references · 18

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  4. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  5. 2020Point Transformercited by 1,470
  6. 2021Transformers in Vision: A Surveycited by 1,013
  7. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  8. 2023Point Transformer V3: Simpler, Faster, Strongercited by 589
  9. 2021CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windowscited by 527
  10. 2022Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizationscited by 363
  11. 2021AdaViT: Adaptive Tokens for Efficient Vision Transformercited by 362
  12. 2023Token Merging for Fast Stable Diffusioncited by 224
6 more
  1. 2021Adaptive Token Sampling For Efficient Vision Transformerscited by 183
  2. 2021Patch Slimming for Efficient Vision Transformerscited by 133
  3. 2024Fit and Prune: Fast and Training-free Visual Token Pruning for Multi-modal Large Language Modelscited by 126
  4. 2023SparseViT: Revisiting Activation Sparsity for Efficient High-Resolution Vision Transformercited by 60
  5. 2024vid-TLDR: Training Free Token merging for Light-weight Video Transformercited by 41
  6. 2025REOrdering Patches Improves Vision Modelscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Similarity-Aware Token Pruning: Your VLM but Faster
  2. 2023No Token Left Behind: Efficient Vision Transformer via Dynamic Token Idling
  3. 2026RAPID: Layer-Wise Redundancy-Aware Pruning and Importance-Driven Token Merging for Efficient ViT
  4. 2026Scaling Vision Transformers Without Training: Prune-and-Merge Token Compression
  5. 2023Token Fusion: Bridging the Gap between Token Pruning and Token Merging
  6. 2023PPT: Token Pruning and Pooling for Efficient Vision Transformers
  7. 2024ImagePiece: Content-aware Re-tokenization for Efficient Image Recognition
  8. 2022Beyond Attentive Tokens: Incorporating Token Importance and Diversity for Efficient Vision Transformers