mlsanityData to 14 Oct 2026

Do Newer Lightweight CNNs Perform Better Under Resource Constraints? A Controlled Multigenerational Study of Architecture, Initialization, Training Budget, and Efficiency

arXiv 2607.01984 ↗First posted 2 Jul 2026cs.LG cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Newer lightweight convolutional neural networks are often presented as improving predictive performance and deployment efficiency, but such claims require controlled evaluation. This study compares nine lightweight CNN model packages across CIFAR-10, CIFAR-100, and Tiny ImageNet under a shared downstream protocol. We report top-1 accuracy, macro F1, top-5 accuracy, parameter count, FP32 storage, GMACs, batch-size-1 latency on an NVIDIA L4 and AMD Ryzen 5 5500U CPU, peak PyTorch CUDA allocated tensor memory, and point estimate Pareto frontiers. EfficientNetV2-S achieves the highest observed top-1 accuracy on CIFAR-10 and CIFAR-100 at 97.57% and 86.98%, while RepViT-M1.0 leads Tiny ImageNet at 79.87%. EfficientNet-B0 remains within 0.22, 0.85, and 1.79 percentage points of the best result on the three datasets while using approximately 79% fewer parameters and 86% fewer GMACs than EfficientNetV2-S. It also appears on every evaluated accuracy and resource Pareto frontier, making it the most consistently competitive intermediate-budget option. MobileNetV3-Small has the lowest GMAC count, is the fastest model under both CPU thread settings, and records higher observed accuracy than MobileNetV4-Conv-S on all three datasets. Under random initialization, it leads MobileNetV4-Conv-S by 2.55, 1.76, and 0.99 points, with paired test-set intervals excluding zero for the fixed trained models. EfficientNet-B0 remains 3.29, 10.10, and 17.54 points below its pretrained counterpart after 100 epochs of scratch training, despite requiring about five times the recorded training time. SqueezeNet1.1 has the fewest parameters and lowest peak CUDA allocation, but substantially weaker accuracy. Latency rankings differ sharply between the L4 and CPU environments, showing that GMACs alone do not predict measured inference performance. Overall, newer designs provide selective rather than universal gains

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 35 references that are in the map. First posted on arXiv 2 Jul 2026.

Builds on

22 of 35 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  3. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  4. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  5. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  6. 2019Searching for MobileNetV3cited by 1,994
  7. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,136
  8. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,135
  9. 2018Do Better ImageNet Models Transfer Better?cited by 565
  10. 2018Rethinking ImageNet Pre-trainingcited by 377
  11. 2023FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterizationcited by 162
  12. 2023Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networkscited by 130
10 more
  1. 2023RepViT: Revisiting Mobile CNN From ViT Perspectivecited by 123
  2. 2022MobileOne: An Improved One millisecond Mobile Backbonecited by 109
  3. 2024MobileNetV4 -- Universal Models for the Mobile Ecosystemcited by 99
  4. 2023Battle of the Backbones: A Large-Scale Comparison of Pretrained Models across Computer Vision Taskscited by 72
  5. 2024Rewrite the Starscited by 55
  6. 2025LSNet: See Large, Focus Smallcited by 15
  7. 2024Which Backbone to Use: A Resource-efficient Domain Specific Comparison for Computer Visioncited by 6
  8. 2025Comparative Analysis of Lightweight CNNs for Resource-Constrained Devices: Predictive Performance, Efficiency Trade-offs, and Initialization Effectscited by 3
  9. 2025UniConvNet: Expanding Effective Receptive Field while Maintaining Asymptotically Gaussian Distribution for ConvNets of Any Scalecited by 3
  10. 2024Vision Backbone Efficient Selection for Image Classification in Low-Data Regimescited by 2

Built on by

Not cited yet.