mlsanityData to 7 Oct 2026

Agglomerative Token Clustering

arXiv 2409.11923 ↗First posted 18 Sep 2024cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

We present Agglomerative Token Clustering (ATC), a novel token merging method that consistently outperforms previous token merging and pruning methods across image classification, image synthesis, and object detection & segmentation tasks. ATC merges clusters through bottom-up hierarchical clustering, without the introduction of extra learnable parameters. We find that ATC achieves state-of-the-art performance across all tasks, and can even perform on par with prior state-of-the-art when applied off-the-shelf, i.e. without fine-tuning. ATC is particularly effective when applied with low keep rates, where only a small fraction of tokens are kept and retaining task performance is especially difficult.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 60 of its 74 references that are in the map. First posted on arXiv 18 Sep 2024.

Builds on

60 of 74 references.

References · 60

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2017Mask R-CNNcited by 7,259
  7. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  8. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  9. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,379
  10. 2022Token Merging: Your ViT But Fastercited by 995
  11. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 849
  12. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 723
38 more
  1. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
  2. 2022Vision Transformer Adapter for Dense Predictionscited by 564
  3. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 562
  4. 2022GroupViT: Semantic Segmentation Emerges from Text Supervisioncited by 529
  5. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  6. 2021Rethinking Spatial Dimensions of Vision Transformerscited by 361
  7. 2021AdaViT: Adaptive Tokens for Efficient Vision Transformercited by 360
  8. 2021AdaViT: Adaptive Vision Transformers for Efficient Image Recognitioncited by 228
  9. 2023Patch n' Pack: NaViT, a Vision Transformer for any Aspect Ratio and Resolutioncited by 226
  10. 2023Token Merging for Fast Stable Diffusioncited by 220
  11. 2021Chasing Sparsity in Vision Transformers: An End-to-End Explorationcited by 181
  12. 2021Adaptive Token Sampling For Efficient Vision Transformerscited by 181
  13. 2021Evo-ViT: Slow-Fast Token Evolution for Dynamic Vision Transformercited by 175
  14. 2021SPViT: Enabling Faster Vision Transformers via Soft Token Pruningcited by 165
  15. 2022FlexiViT: One Model for All Patch Sizescited by 151
  16. 2021FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformercited by 148
  17. 2021Not All Images are Worth 16x16 Words: Dynamic Transformers for Efficient Image Recognitioncited by 143
  18. 2021TransFG: A Transformer Architecture for Fine-grained Recognitioncited by 131
  19. 2021IA-RED2: Interpretability-Aware Redundancy Reduction for Vision Transformerscited by 127
  20. 2022Revisiting Weakly Supervised Pre-Training of Visual Perception Modelscited by 116
  21. 2022Not All Tokens Are Equal: Human-centric Visual Analysis via Token Clustering Transformercited by 99
  22. 2020Machine Learning in Python: Main developments and technology trends in data science, machine learning, and artificial intelligencecited by 81
  23. 2023Joint Token Pruning and Squeezing Towards More Aggressive Compression of Vision Transformerscited by 79
  24. 2022Vision Transformer Slimming: Multi-Dimension Searching in Continuous Optimization Spacecited by 66
  25. 2022Learning to Merge Tokens in Vision Transformerscited by 64
  26. 2022CF-ViT: A General Coarse-to-Fine Method for Vision Transformercited by 63
  27. 2023SparseViT: Revisiting Activation Sparsity for Efficient High-Resolution Vision Transformercited by 60
  28. 2023Which Tokens to Use? Investigating Token Reduction in Vision Transformerscited by 59
  29. 2021Less is More: Pay Less Attention in Vision Transformerscited by 53
  30. 2023Revisiting Token Pruning for Object Detection and Instance Segmentationcited by 50
  31. 2022Beyond Attentive Tokens: Incorporating Token Importance and Diversity for Efficient Vision Transformerscited by 47
  32. 2018Faster k-Medoids Clustering: Improving the PAM, CLARA, and CLARANS Algorithmscited by 44
  33. 2021Feature Fusion Vision Transformer for Fine-Grained Visual Categorizationcited by 40
  34. 2021A Unified Pruning Framework for Vision Transformerscited by 40
  35. 2022CP-ViT: Cascade Vision Transformer Pruning via Progressive Sparsity Predictioncited by 39
  36. 2023PPT: Token Pruning and Pooling for Efficient Vision Transformerscited by 37
  37. 2022Q-ViT: Fully Differentiable Quantization for Vision Transformercited by 36
  38. 2021RAMS-Trans: Recurrent Attention Multi-scale Transformer forFine-grained Image Recognitioncited by 34

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 5; 2026: 7.

13 papers.

Sort:

By area:

  1. 2025Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodalitycited by 49
  2. 2024Importance-Based Token Merging for Efficient Image and Video Generationcited by 20
  3. 2025Sparse Imagination for Efficient Visual World Model Planningcited by 7
  4. 2025LookWhere? Efficient Visual Recognition by Learning Where to Look and What to See from Self-Supervisioncited by 5
  5. 2025PTCMIL: Multiple Instance Learning via Prompt Token Clustering for Whole Slide Image Analysiscited by 2
  6. 2026Foveated Diffusion: Efficient Spatially Adaptive Image and Video Generationcited by 2
  7. 2026MPM: Mutual Pair Merging for Efficient Vision Transformerscited by 2
  8. 2025Local Representative Token Guided Merging for Text-to-Image Generationcited by 1
  9. 2026LookWhen? Fast Video Recognition by Learning When, Where, and What to Computecited by 1
  10. 2026Variable-Length Tokenization via Learnable Global Merging for Diffusion Transformerscited by 0
  11. 2026When Structured Sparse Autoencoders Learn Consistent Concepts Across Modalitiescited by 0
  12. 2026Importance-Aware OBS Pruning for Diffusion Modelscited by 0
1 more
  1. 2026Decoder-Agnostic Token Merging for Vision Transformers: A Systematic Study of G2TMcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Semantic Equitable Clustering: A Simple and Effective Strategy for Clustering Vision Tokens
  2. 2026VLM4Cluster: Benchmarking Deep Clustering In the Era of Vision-Language Pre-training
  3. 2026Hierarchical Textual Knowledge for Enhanced Image Clustering
  4. 2024Subobject-level Image Tokenization
  5. 2026Language-Assisted Image Clustering Guided by Discriminative Relational Signals and Adaptive Semantic Centers
  6. 2026Clustering and Token Denoising for Faster and More Robust VLMs
  7. 2023Image Clustering Conditioned on Text Criteria
  8. 2024DiFiC: Your Diffusion Model Holds the Secret to Fine-Grained Clustering