SpiralFovea: Input-Adaptive Foveated Tokenization as a Third Lever of Resource-Adaptive Inference
On the map: Mixture-of-experts → Vision mixture-of-experts
Abstract (arXiv)
Most adaptive-inference techniques for foundation models change what the model does - early exit, MoE routing, KV-cache compression, dynamic attention sparsity. The input that hits the backbone, however, remains a fixed-grid tokenisation indifferent to image content. We argue that this is a missed lever. We present SpiralFovea, a parameter-free, input-adaptive tokeniser in which token identity, location, scale, and count are all functions of local visual entropy and selection completes before any backbone parameter is queried. Around content-driven hotspot anchors, multi-scale spiral rings produce <= 78 patches that replace the standard 196-patch ViT grid at the input stage. Across four canonical fine-grained benchmarks, SpiralFovea yields +1.7-2.1 pp accuracy with a 60% reduction in input tokens, an 84% reduction in self-attention FLOPs at every transformer layer, and 18-29% throughput gains over the matched static tokenisation baseline. A controlled ablation on CUB-200-2011 Genus across four backbones reveals a clean diagnostic: the gain magnitude tracks inversely with the strength of the backbone's whole-image positional prior, isolating self-supervised foundation models as the regime where input-adaptive tokenisation is most valuable.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts. Builds on 12 of its 20 references that are in the map. First posted on arXiv 1 Jul 2026.
Builds on
12 of 20 references.
References · 12
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
- 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
- 2015Spatial Transformer Networkscited by 1,423
- 2024VMamba: Visual State Space Modelcited by 955
- 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 849
- 2018Rotation Equivariant CNNs for Digital Pathologycited by 420
- 2022Not All Patches are What You Need: Expediting Vision Transformers via Token Reorganizationscited by 359
- 2022Vision Transformer with Deformable Attentioncited by 293
- 2015Large-scale Classification of Fine-Art Paintings: Learning The Right Metric on The Right Featurecited by 181
- 2021Adaptive Token Sampling For Efficient Vision Transformerscited by 181
- 2018Learning to Zoom: a Saliency-Based Sampling Layer for Neural Networkscited by 56
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Foveated Compression: Selective High-Resolution Preservation for Token-Efficient VLMs
- 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects
- 2026FOVEA: Focused On-Demand Visual Evidence Adaptation for Cache-Friendly Multimodal Speculative Decoding
- 2026ApET: Approximation-Error Guided Token Compression for Efficient VLMs
- 2026EviViT: Evidence-Adaptive Vision Transformers for Fine-Grained Perception
- 2025Efficient Vision-Language Reasoning via Adaptive Token Pruning
- 2026One Layer's Trash is Another Layer's Treasure: Adaptive Layer-wise Visual Token Selection in LVLMs
- 2026Adaptive Visual Token Reduction for Accelerated Image Understanding