mlsanityData to 14 Oct 2026

Hybrid Convolution and Vision Transformer NAS Search Space for TinyML Image Classification

arXiv 2511.02992 ↗First posted 4 Nov 2025cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Hybrids of Convolutional Neural Network (CNN) and Vision Transformer (ViT) have outperformed pure CNN or ViT architecture. However, since these architectures require large parameters and incur large computational costs, they are unsuitable for tinyML deployment. This paper introduces a new hybrid CNN-ViT search space for Neural Architecture Search (NAS) to find efficient hybrid architectures for image classification. The search space covers hybrid CNN and ViT blocks to learn local and global information, as well as the novel Pooling block of searchable pooling layers for efficient feature map reduction. Experimental results on the CIFAR10 dataset show that our proposed search space can produce hybrid CNN-ViT architectures with superior accuracy and inference speed to ResNet-based tinyML models under tight model size constraints.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 37 references that are in the map. First posted on arXiv 4 Nov 2025.

Builds on

27 of 37 references.

References · 27

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  7. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  9. 2019Searching for MobileNetV3cited by 1,994
  10. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  11. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  12. 2020Linformer: Self-Attention with Linear Complexitycited by 1,474
15 more
  1. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 909
  2. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 787
  3. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
  4. 2021MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformercited by 642
  5. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  6. 2021Conditional Positional Encodings for Vision Transformerscited by 406
  7. 2020MCUNet: Tiny Deep Learning on IoT Devicescited by 273
  8. 2022Separable Self-attention for Mobile Vision Transformerscited by 163
  9. 2021MLPerf Tiny Benchmarkcited by 124
  10. 2022EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformerscited by 116
  11. 2018A Flexible Framework for Multi-Objective Bayesian Optimization using Random Scalarizationscited by 99
  12. 2020μNAS: Constrained Neural Architecture Search for Microcontrollerscited by 55
  13. 2021Searching the Search Space of Vision Transformercited by 41
  14. 2022Hardware Accelerator and Neural Network Co-Optimization for Ultra-Low-Power Audio Processing Devicescited by 8
  15. 2023MLonMCU: TinyML Benchmarking with Fast Retargetingcited by 5

Built on by

Not cited yet.