mlsanityData to 14 Oct 2026

EfficientNetV2: Smaller Models and Faster Training

arXiv 2104.00298 ↗First posted 1 Apr 2021cs.CVcited by 1,135

Not on the map. Why?

Abstract (arXiv)

This paper introduces EfficientNetV2, a new family of convolutional networks that have faster training speed and better parameter efficiency than previous models. To develop this family of models, we use a combination of training-aware neural architecture search and scaling, to jointly optimize training speed and parameter efficiency. The models were searched from the search space enriched with new ops such as Fused-MBConv. Our experiments show that EfficientNetV2 models train much faster than state-of-the-art models while being up to 6.8x smaller. Our training can be further sped up by progressively increasing the image size during training, but it often causes a drop in accuracy. To compensate for this accuracy drop, we propose to adaptively adjust regularization (e.g., dropout and data augmentation) as well, such that we can achieve both fast training and good accuracy. With progressive learning, our EfficientNetV2 significantly outperforms previous models on ImageNet and CIFAR/Cars/Flowers datasets. By pretraining on the same ImageNet21k, our EfficientNetV2 achieves 87.3% top-1 accuracy on ImageNet ILSVRC2012, outperforming the recent ViT by 2.0% accuracy while training 5x-11x faster using the same computing resources. Code will be available at https://github.com/google/automl/tree/master/efficientnetv2.

Key facts. Cited by 1,135 papers in the map (data through 14 Oct 2026). Builds on 38 of its 50 references that are in the map. First posted on arXiv 1 Apr 2021.

Builds on

38 of 50 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  5. 2016Densely Connected Convolutional Networkscited by 6,720
  6. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  7. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  8. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  9. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  10. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  11. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  12. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
26 more
  1. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  2. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,101
  3. 2019EfficientDet: Scalable and Efficient Object Detectioncited by 1,020
  4. 2016Deep Networks with Stochastic Depthcited by 979
  5. 2020Designing Network Design Spacescited by 960
  6. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 943
  7. 2019Big Transfer (BiT): General Visual Representation Learningcited by 769
  8. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
  9. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 605
  10. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 485
  11. 2020ResNeSt: Split-Attention Networkscited by 445
  12. 2021Bottleneck Transformers for Visual Recognitioncited by 364
  13. 2021High-Performance Large-Scale Image Recognition Without Normalizationcited by 360
  14. 2019Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentationcited by 260
  15. 2019Fixing the train-test resolution discrepancycited by 209
  16. 2021Revisiting ResNets: Improved Training and Scaling Strategiescited by 158
  17. 2020TResNet: High Performance GPU-Dedicated Architecturecited by 119
  18. 2019MixConv: Mixed Depthwise Convolutional Kernelscited by 108
  19. 2021LambdaNetworks: Modeling Long-Range Interactions Without Attentioncited by 107
  20. 2020Shortformer: Better Language Modeling using Shorter Inputscited by 76
  21. 2020MobileDets: Searching for Object Detection Architectures for Mobile Acceleratorscited by 55
  22. 2020Fixing the train-test resolution discrepancy: FixEfficientNetcited by 31
  23. 2020Accelerator-aware Neural Network Design using AutoMLcited by 30
  24. 2021Searching for Fast Model Families on Datacenter Acceleratorscited by 22
  25. 2019Mix & Match: training convnets with mixed image sizes for improved accuracy, speed and scale resiliencycited by 12
  26. 2019PDA: Progressive Data Augmentation for General Robustness of Deep Neural Networkscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 70; 2022: 168; 2023: 230; 2024: 249; 2025: 244; 2026: 174.

1,135 papers.

Sort:

By area:

  1. 2022A ConvNet for the 2020scited by 3,579
  2. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,056
  3. 2021Scaling Vision Transformerscited by 999
  4. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 982
  5. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 712
  6. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 569
  7. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 569
  8. 2022InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutionscited by 482
  9. 2022MaxViT: Multi-Axis Vision Transformercited by 476
  10. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  11. 2022DeiT III: Revenge of the ViTcited by 419
  12. 2021OCR-free Document Understanding Transformercited by 407
38 more
  1. 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 404
  2. 2021Early Convolutions Help Transformers See Bettercited by 396
  3. 2021Pay Attention to MLPscited by 368
  4. 2021ResNet strikes back: An improved training procedure in timmcited by 368
  5. 2021CMT: Convolutional Neural Networks Meet Vision Transformerscited by 310
  6. 2021ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generationcited by 260
  7. 2022CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformerscited by 227
  8. 2024MambaVision: A Hybrid Mamba-Transformer Vision Backbonecited by 224
  9. 2022Visual Attention Networkcited by 210
  10. 2021Vision Transformers are Robust Learnerscited by 207
  11. 2022UniFormer: Unifying Convolution and Self-attention for Visual Recognitioncited by 206
  12. 2022UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learningcited by 184
  13. 2021Combined Scaling for Zero-shot Transfer Learningcited by 182
  14. 2023AM-RADIO: Agglomerative Vision Foundation Model -- Reduce All Domains Into Onecited by 169
  15. 2021VOLO: Vision Outlooker for Visual Recognitioncited by 165
  16. 2023FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterizationcited by 162
  17. 2023EfficientAD: Accurate Visual Anomaly Detection at Millisecond-Level Latenciescited by 160
  18. 2022Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?cited by 155
  19. 2022FlexiViT: One Model for All Patch Sizescited by 152
  20. 2022MetaFormer Baselines for Visioncited by 132
  21. 2023Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networkscited by 130
  22. 2022EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformerscited by 116
  23. 2022MobileOne: An Improved One millisecond Mobile Backbonecited by 109
  24. 2021AS-MLP: An Axial Shifted MLP Architecture for Visioncited by 101
  25. 2022Inception Transformercited by 99
  26. 2022A Generalist Framework for Panoptic Segmentation of Images and Videoscited by 99
  27. 2022Global Context Vision Transformerscited by 98
  28. 2024MedMamba: Vision Mamba for Medical Image Classificationcited by 88
  29. 2023InceptionNeXt: When Inception Meets ConvNeXtcited by 86
  30. 2022Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challengescited by 84
  31. 2022SimpleRecon: 3D Reconstruction Without 3D Convolutionscited by 83
  32. 2024The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speechcited by 83
  33. 2022TopFormer: Token Pyramid Transformer for Mobile Semantic Segmentationcited by 82
  34. 2022Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognitioncited by 81
  35. 2023Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Modelscited by 79
  36. 2023Cinematic Mindscapes: High-quality Video Reconstruction from Brain Activitycited by 71
  37. 2023Generalized Large-Scale Data Condensation via Various Backbone and Statistical Matchingcited by 69
  38. 2022The Effects of Regularization and Data Augmentation are Class Dependentcited by 67

50 most cited of 1,135 All 1,135 (JSON)