# Exploring the Synergies of Hybrid CNNs and ViTs Architectures for Computer Vision: A survey

arXiv: https://arxiv.org/abs/2402.02941 · HTML: https://mlsanity.com/paper/2402.02941/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 81 of its 128 references that are in the map.
- First posted on arXiv 5 Feb 2024.

## Builds on (most cited first, 30 of 81)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Linformer: Self-Attention with Linear Complexity](https://mlsanity.com/paper/2006.04768/) (2020)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [Scaling Vision Transformers](https://mlsanity.com/paper/2106.04560/) (2021)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [Segmenter: Transformer for Semantic Segmentation](https://mlsanity.com/paper/2105.05633/) (2021)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)

## Built on by (most cited first, 13 of 13)

- [Hypergraph Vision Transformers: Images are More than Nodes, More than Edges](https://mlsanity.com/paper/2504.08710/) (2025)
- [Convolutional Neural Networks and Vision Transformers for Fashion MNIST Classification: A Literature Review](https://mlsanity.com/paper/2406.03478/) (2024)
- [An Attention-Guided Deep Learning Approach for Classifying 39 Skin Lesion Types](https://mlsanity.com/paper/2501.05991/) (2025)
- [A Retrospective Systematic Study on Hierarchical Sparse Query Transformer-assisted Ultrasound Screening for Early Hepatocellular Carcinoma](https://mlsanity.com/paper/2502.03772/) (2025)
- [Dual sparse training framework: inducing activation map sparsity via Transformed ℓ1 regularization](https://mlsanity.com/paper/2405.19652/) (2024)
- [iiANET: Inception Inspired Attention Hybrid Network for efficient Long-Range Dependency](https://mlsanity.com/paper/2407.07603/) (2024)
- [KonvLiNA: Integrating Kolmogorov-Arnold Network with Linear Nystr"om Attention for feature fusion in Crop Field Detection](https://mlsanity.com/paper/2408.13160/) (2024)
- [Data-driven tool wear prediction in milling, based on a process-integrated single-sensor approach](https://mlsanity.com/paper/2412.19950/) (2024)
- [DFYP: A Dynamic Fusion Framework with Spectral Channel Attention and Adaptive Operator learning for Crop Yield Prediction](https://mlsanity.com/paper/2507.05849/) (2025)
- [CNN-ViT Hybrid for Pneumonia Detection: Theory and Empiric on Limited Data without Pretraining](https://mlsanity.com/paper/2509.08586/) (2025)
- [MedCondDiff: Lightweight, Robust, Semantically Guided Diffusion for Medical Image Segmentation](https://mlsanity.com/paper/2512.00350/) (2025)
- [Federated Vision Transformer with Adaptive Focal Loss for Medical Image Classification](https://mlsanity.com/paper/2602.01633/) (2026)
- [Enhancing Shrimp Disease Detection via Deep Learning and Data Refinement for Resilient Aquaculture](https://mlsanity.com/paper/2609.23397/) (2026)

## Abstract (arXiv)

The hybrid of Convolutional Neural Network (CNN) and Vision Transformers (ViT) architectures has emerged as a groundbreaking approach, pushing the boundaries of computer vision (CV). This comprehensive review provides a thorough examination of the literature on state-of-the-art hybrid CNN-ViT architectures, exploring the synergies between these two approaches. The main content of this survey includes: (1) a background on the vanilla CNN and ViT, (2) systematic review of various taxonomic hybrid designs to explore the synergy achieved through merging CNNs and ViTs models, (3) comparative analysis and application task-specific synergy between different hybrid architectures, (4) challenges and future directions for hybrid models, (5) lastly, the survey concludes with a summary of key findings and recommendations. Through this exploration of hybrid CV architectures, the survey aims to serve as a guiding resource, fostering a deeper understanding of the intricate dynamics between CNNs and ViTs and their collective impact on shaping the future of CV architectures.