mlsanityData to 7 Oct 2026

Differentiable Hierarchical Visual Tokenization

arXiv 2511.02652 ↗First posted 4 Nov 2025cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Vision Transformers rely on fixed patch tokens that ignore the spatial and semantic structure of images. In this work, we introduce an end-to-end differentiable tokenizer that adapts to image content with pixel-level granularity while remaining backward-compatible with existing architectures for retrofitting pretrained models. Our method uses hierarchical model selection with information criteria to provide competitive performance in both image-level classification and dense-prediction tasks, and even supports out-of-the-box raster-to-vector conversion.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 50 of its 74 references that are in the map. First posted on arXiv 4 Nov 2025.

Builds on

50 of 74 references.

References · 50

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  8. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  9. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  10. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  11. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  12. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
38 more
  1. 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
  2. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,507
  3. 2020Point Transformercited by 1,462
  4. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  5. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  6. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  7. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  8. 2022Token Merging: Your ViT But Fastercited by 995
  9. 2020Object-Centric Learning with Slot Attentioncited by 865
  10. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  11. 2019Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingcited by 748
  12. 2021AST: Audio Spectrogram Transformercited by 634
  13. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 562
  14. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  15. 2022DeiT III: Revenge of the ViTcited by 417
  16. 2020Are we done with ImageNet?cited by 329
  17. 2022Vision Transformer with Deformable Attentioncited by 293
  18. 2022Self-Supervised Transformers for Unsupervised Object Discovery using Normalized Cutcited by 166
  19. 2023BiFormer: Vision Transformer with Bi-Level Routing Attentioncited by 150
  20. 2022SegViT: Semantic Segmentation with Plain Vision Transformerscited by 105
  21. 2022Towards Layer-wise Image Vectorizationcited by 77
  22. 2023A Comprehensive Survey on Applications of Transformers for Deep Learning Taskscited by 70
  23. 2021SeMask: Semantically Masked Transformers for Semantic Segmentationcited by 55
  24. 2013Estimating or Propagating Gradients Through Stochastic Neuronscited by 54
  25. 2021DPT: Deformable Patch-based Transformer for Visual Recognitioncited by 43
  26. 2022CLIP Itself is a Strong Fine-tuner: Achieving 85.7% and 88.0% Top-1 Accuracy with ViT-B and ViT-L on ImageNetcited by 35
  27. 2023Vision Transformers with Mixed-Resolution Tokenizationcited by 35
  28. 2022SPSN: Superpixel Prototype Sampling Network for RGB-D Salient Object Detectioncited by 26
  29. 2024Subobject-level Image Tokenizationcited by 20
  30. 2024Multi-Scale Representations by Varying Window Attention for Semantic Segmentationcited by 20
  31. 2023MSViT: Dynamic Mixed-Scale Tokenization for Vision Transformerscited by 19
  32. 2022Region Rebalance for Long-Tailed Semantic Segmentationcited by 17
  33. 2021Efficient Self-Ensemble for Semantic Segmentationcited by 16
  34. 2022CAR: Class-aware Regularizations for Semantic Segmentationcited by 14
  35. 2024SPFormer: Enhancing Vision Transformer with Superpixel Representationcited by 13
  36. 2023Superpixel Transformers for Efficient Semantic Segmentationcited by 12
  37. 2024A Spitting Image: Modular Superpixel Tokenization in Vision Transformerscited by 7
  38. 2024Superpixel Tokenization for Vision Transformers: Preserving Semantic Integrity in Visual Tokenscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026A Hyperbolic Perspective on Hierarchical Structure in Object-Centric Scene Representationscited by 1
  2. 2026Do Vision Transformers Need All-to-All Attention? Global Communication Through Elastic Learned Corescited by 0
  3. 2026Variable-Granularity Tokenization for High-Resolution Object Detectioncited by 0