mlsanityData to 13 Oct 2026

Vision Transformers in 2022: An Update on Tiny ImageNet

arXiv 2205.10660 ↗First posted 21 May 2022cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

The recent advances in image transformers have shown impressive results and have largely closed the gap between traditional CNN architectures. The standard procedure is to train on large datasets like ImageNet-21k and then finetune on ImageNet-1k. After finetuning, researches will often consider the transfer learning performance on smaller datasets such as CIFAR-10/100 but have left out Tiny ImageNet. This paper offers an update on vision transformers' performance on Tiny ImageNet. I include Vision Transformer (ViT) , Data Efficient Image Transformer (DeiT), Class Attention in Image Transformer (CaiT), and Swin Transformers. In addition, Swin Transformers beats the current state-of-the-art result with a validation accuracy of 91.35%. Code is available here: https://github.com/ehuynh1106/TinyImageNet-Transformers

Key facts. Cited by 13 papers in the map (data through 13 Oct 2026). Builds on 16 of its 22 references that are in the map. First posted on arXiv 21 May 2022.

Builds on

16 of 22 references.

References · 16

  1. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,111
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,476
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,170
  4. 2020Training data-efficient image transformers & distillation through attentioncited by 4,574
  5. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,882
  6. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,411
  7. 2017Random Erasing Data Augmentationcited by 1,238
  8. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,219
  9. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  10. 2021Going deeper with Image Transformerscited by 762
  11. 2018AutoAugment: Learning Augmentation Policies from Datacited by 730
  12. 2021ImageNet-21K Pretraining for the Massescited by 611
4 more
  1. 2022DeiT III: Revenge of the ViTcited by 417
  2. 2021ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networkscited by 293
  3. 2021Vision Transformer for Small-Size Datasetscited by 122
  4. 2022MiniViT: Compressing Vision Transformers with Weight Multiplexingcited by 88

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 1; 2025: 5; 2026: 2.

13 papers.

Sort:

By area:

  1. 2023When Prompt-based Incremental Learning Does Not Meet Strong Pretrainingcited by 41
  2. 2023Learn to Accumulate Evidence from All Training Samples: Theory and Practicecited by 28
  3. 2022MPCViT: Searching for Accurate and Efficient MPC-Friendly Vision Transformer with Heterogeneous Attentioncited by 19
  4. 2022Diffuser: Efficient Transformers with Multi-hop Attention Diffusion for Long Sequencescited by 12
  5. 2023Astroformer: More Data Might not be all you need for Classificationcited by 6
  6. 2024Towards Optimal Trade-offs in Knowledge Distillation for CNNs and Vision Transformers at the Edgecited by 4
  7. 2025CNN and ViT Efficiency Study on Tiny ImageNet and DermaMNIST Datasetscited by 4
  8. 2025A Model Zoo of Vision Transformerscited by 3
  9. 2025Unleashing Foundation Vision Models: Adaptive Transfer for Diverse Data-Limited Scientific Domainscited by 2
  10. 2025Generalized Regularized Evidential Deep Learning Models: Theory and Comprehensive Evaluationcited by 2
  11. 2025Autoguided Online Data Curation for Diffusion Model Trainingcited by 1
  12. 2026Batch Normalization Amplifies Memorization and Privacy Riskscited by 1
1 more
  1. 2026Enhancing Federated Quadruplet Learning: Stochastic Client Selection and Embedding Stability Analysiscited by 0