mlsanityData to 7 Oct 2026

UtVAA: Ultra-tiny Vision Transformer with Affix Attention for Mobile Image Classification

arXiv 2606.14735 ↗First posted 2 Jun 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Vision Transformers (ViTs) have demonstrated strong representation capability in image classification. However, their quadratic self-attention complexity and large parameter counts limit deployment on resource-constrained mobile and edge devices. This paper introduces UtVAA, an ultra-tiny Vision Transformer architecture designed for efficient visual recognition under strict computational budgets. It incorporates a novel Affix Attention block that combines depthwise-pointwise local feature extraction, linear self-attention, coordinate attention for spatial dependency modelling, and a lightweight ternary fusion strategy to integrate local and global representations. In addition, Dilated Bottleneck blocks expand the receptive field using dilated depthwise separable convolutions while maintaining low FLOPs and stable optimisation through residual connections. UtVAA is implemented in scalable Tiny, Medium, and Large variants, with the smallest model containing 204.67K parameters and 53.95M FLOPs. Experimental results on CIFAR-10, CIFAR-100, PlantVillage-Tomato and SLIF-Tomato datasets show that UtVAA achieves competitive accuracy within a sub-million-parameter regime. Overall, the results demonstrate that transformer-based vision models can be redesigned into ultra-tiny architectures without significant loss in discriminative performance, making UtVAA suitable for mobile and edge deployment. Code is available at https://github.com/romiyal/UtVAA

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 35 of its 72 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

35 of 72 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2016Densely Connected Convolutional Networkscited by 6,691
  6. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  7. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  8. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  9. 2017Squeeze-and-Excitation Networkscited by 4,763
  10. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  11. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  12. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
23 more
  1. 2018CBAM: Convolutional Block Attention Modulecited by 2,463
  2. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
  3. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  4. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,128
  5. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
  6. 2021MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformercited by 636
  7. 2019ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networkscited by 572
  8. 2020A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospectscited by 396
  9. 2021Rethinking Spatial Dimensions of Vision Transformerscited by 361
  10. 2021Coordinate Attention for Efficient Mobile Network Designcited by 239
  11. 2021Attention Mechanisms in Computer Vision: A Surveycited by 232
  12. 2024Lightweight Deep Learning for Resource-Constrained Environments: A Surveycited by 38
  13. 2024Lightweight Pixel Difference Networks for Efficient Visual Representation Learningcited by 17
  14. 2024Depth-Wise Convolutions in Vision Transformers for Efficient Training on Small Datasetscited by 14
  15. 2024CAS-ViT: Convolutional Additive Self-attention Vision Transformers for Efficient Mobile Applicationscited by 12
  16. 2024Hybrid Convolutional and Attention Network for Hyperspectral Image Denoisingcited by 9
  17. 2022Complexity-Driven CNN Compression for Resource-constrained Edge AIcited by 8
  18. 2023Enhancing Neural Architecture Search with Multiple Hardware Constraints for Deep Learning Model Deployment on Tiny IoT Devicescited by 7
  19. 2023PSLT: A Light-weight Vision Transformer with Ladder Self-Attention and Progressive Shiftcited by 5
  20. 2025From Tiny Machine Learning to Tiny Deep Learning: A Surveycited by 5
  21. 2025UltraLBM-UNet: Ultralight Bidirectional Mamba-based Model for Skin Lesion Segmentationcited by 3
  22. 2025Building Efficient Lightweight CNN Modelscited by 1
  23. 2025A Lightweight Convolution and Vision Transformer integrated model with Multi-scale Self-attention Mechanismcited by 1

Built on by

Not cited yet.