mlsanityData to 7 Oct 2026

SpiralFovea: Input-Adaptive Foveated Tokenization as a Third Lever of Resource-Adaptive Inference

arXiv 2607.00780 ↗First posted 1 Jul 2026cs.CVcited by 1

On the map: Mixture-of-experts → Vision mixture-of-experts

also in 1 more
Abstract (arXiv)

Most adaptive-inference techniques for foundation models change what the model does - early exit, MoE routing, KV-cache compression, dynamic attention sparsity. The input that hits the backbone, however, remains a fixed-grid tokenisation indifferent to image content. We argue that this is a missed lever. We present SpiralFovea, a parameter-free, input-adaptive tokeniser in which token identity, location, scale, and count are all functions of local visual entropy and selection completes before any backbone parameter is queried. Around content-driven hotspot anchors, multi-scale spiral rings produce <= 78 patches that replace the standard 196-patch ViT grid at the input stage. Across four canonical fine-grained benchmarks, SpiralFovea yields +1.7-2.1 pp accuracy with a 60% reduction in input tokens, an 84% reduction in self-attention FLOPs at every transformer layer, and 18-29% throughput gains over the matched static tokenisation baseline. A controlled ablation on CUB-200-2011 Genus across four backbones reveals a clean diagnostic: the gain magnitude tracks inversely with the strength of the backbone's whole-image positional prior, isolating self-supervised foundation models as the regime where input-adaptive tokenisation is most valuable.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts. Builds on 12 of its 20 references that are in the map. First posted on arXiv 1 Jul 2026.

Builds on

12 of 20 references.

References · 12

  1. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  2. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  3. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  4. 2015Spatial Transformer Networkscited by 1,423
  5. 2024VMamba: Visual State Space Modelcited by 955
  6. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 849
  7. 2018Rotation Equivariant CNNs for Digital Pathologycited by 420
  8. 2022Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizationscited by 359
  9. 2022Vision Transformer with Deformable Attentioncited by 293
  10. 2015Large-scale Classification of Fine-Art Paintings: Learning The Right Metric on The Right Featurecited by 181
  11. 2021Adaptive Token Sampling For Efficient Vision Transformerscited by 181
  12. 2018Learning to Zoom: a Saliency-Based Sampling Layer for Neural Networkscited by 56

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Foveated Compression: Selective High-Resolution Preservation for Token-Efficient VLMs
  2. 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects
  3. 2026FOVEA: Focused On-Demand Visual Evidence Adaptation for Cache-Friendly Multimodal Speculative Decoding
  4. 2026ApET: Approximation-Error Guided Token Compression for Efficient VLMs
  5. 2026EviViT: Evidence-Adaptive Vision Transformers for Fine-Grained Perception
  6. 2025Efficient Vision-Language Reasoning via Adaptive Token Pruning
  7. 2026One Layer's Trash is Another Layer's Treasure: Adaptive Layer-wise Visual Token Selection in LVLMs
  8. 2026Adaptive Visual Token Reduction for Accelerated Image Understanding