# Spectral Vision Transformer for Efficient Tokenization with Limited Data

arXiv: https://arxiv.org/abs/2605.12026 · HTML: https://mlsanity.com/paper/2605.12026/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 61 references that are in the map.
- First posted on arXiv 12 May 2026.

## Builds on (most cited first, 24 of 24)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [Revisiting Batch Normalization For Practical Domain Adaptation](https://mlsanity.com/paper/1603.04779/) (2016)
- [Global Filter Networks for Image Classification](https://mlsanity.com/paper/2107.00645/) (2021)
- [A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises](https://mlsanity.com/paper/2008.09104/) (2020)
- [Frequency-domain MLPs are More Effective Learners in Time Series Forecasting](https://mlsanity.com/paper/2311.06184/) (2023)
- [Surgical Data Science -- from Concepts toward Clinical Translation](https://mlsanity.com/paper/2011.02284/) (2020)
- [Wave-ViT: Unifying Wavelet and Transformers for Visual Representation Learning](https://mlsanity.com/paper/2207.04978/) (2022)
- [Locality Guidance for Improving Vision Transformers on Tiny Datasets](https://mlsanity.com/paper/2207.10026/) (2022)
- [Enhancing Foundation Models for Time Series Forecasting via Wavelet-based Tokenization](https://mlsanity.com/paper/2412.05244/) (2024)
- [Spectral Image Tokenizer](https://mlsanity.com/paper/2412.09607/) (2024)
- [A Review of Predictive and Contrastive Self-supervised Learning for Medical Images](https://mlsanity.com/paper/2302.05043/) (2023)
- [Prior Knowledge-Guided Attention in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2209.03745/) (2022)
- [Denoising Vision Transformer Autoencoder with Spectral Self-Regularization](https://mlsanity.com/paper/2511.12633/) (2025)
- [A Comparative Study of Graph Neural Networks for Shape Classification in Neuroimaging](https://mlsanity.com/paper/2210.16670/) (2022)
- [FreDFT: Frequency Domain Fusion Transformer for Visible-Infrared Object Detection](https://mlsanity.com/paper/2511.10046/) (2025)
- [Maximum Spherical Mean Value (mSMV) Filtering for Whole Brain Quantitative Susceptibility Mapping](https://mlsanity.com/paper/2304.11476/) (2023)
- [Universal Spectral Tokenization via Self-Supervised Panchromatic Representation Learning](https://mlsanity.com/paper/2510.17959/) (2025)
- [Explicitly Increasing Input Information Density for Vision Transformers on Small Datasets](https://mlsanity.com/paper/2210.14319/) (2022)
- [Synthetic Generation and Latent Projection Denoising of Rim Lesions in Multiple Sclerosis](https://mlsanity.com/paper/2505.23353/) (2025)
- [Three-dimensional end-to-end deep learning for brain MRI analysis](https://mlsanity.com/paper/2506.23916/) (2025)

## Abstract (arXiv)

We propose a novel spectral vision transformer architecture for efficient tokenization in limited data, with an emphasis on medical imaging. We outline convenient theoretical properties arising from the choice of basis including spatial invariance and optimal signal-to-noise ratio. We show reduced complexity arising from the spectral projection compared to spatial vision transformers. We show equitable or superior performance with a reduced number of parameters as compared to a variety of models including compact and standard vision transformers, convolutional neural networks with attention, shifted window transformers, multi-layer perceptrons, and logistic regression. We include simulated, public, and clinical data in our analysis and release our code at: verb+github.com/agr78/spectralViT+.