# A survey of the Vision Transformers and their CNN-Transformer based Variants

arXiv: https://arxiv.org/abs/2305.09880 · HTML: https://mlsanity.com/paper/2305.09880/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 24 references that are in the map.
- First posted on arXiv 17 May 2023.

## Builds on (most cited first, 14 of 14)

- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [PVT v2: Improved Baselines with Pyramid Vision Transformer](https://mlsanity.com/paper/2106.13797/) (2021)
- [Transformers in Medical Imaging: A Survey](https://mlsanity.com/paper/2201.09873/) (2022)
- [A Survey of the Recent Architectures of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1901.06032/) (2019)
- [A Survey of Vision-Language Pre-Trained Models](https://mlsanity.com/paper/2202.10936/) (2022)
- [ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyond](https://mlsanity.com/paper/2202.10108/) (2022)
- [Transformers in Remote Sensing: A Survey](https://mlsanity.com/paper/2209.01206/) (2022)
- [DPT: Deformable Patch-based Transformer for Visual Recognition](https://mlsanity.com/paper/2107.14467/) (2021)
- [Recent progress in semantic image segmentation](https://mlsanity.com/paper/1809.10198/) (2018)
- [Segmentation of Shoulder Muscle MRI Using a New Region and Edge based Deep Auto-Encoder](https://mlsanity.com/paper/2108.11720/) (2021)
- [AggPose: Deep Aggregation Vision Transformer for Infant Pose Estimation](https://mlsanity.com/paper/2205.05277/) (2022)
- [Generalised Image Outpainting with U-Transformer](https://mlsanity.com/paper/2201.11403/) (2022)
- [HaloAE: An HaloNet based Local Transformer Auto-Encoder for Anomaly Detection and Localization](https://mlsanity.com/paper/2208.03486/) (2022)
- [A Recent Survey on the Applications of Genetic Programming in Image Processing](https://mlsanity.com/paper/1901.07387/) (2019)

## Built on by (most cited first, 30 of 33)

- [A Recent Survey of Vision Transformers for Medical Image Segmentation](https://mlsanity.com/paper/2312.00634/) (2023)
- [A Comprehensive Survey of Convolutions in Deep Learning: Applications, Challenges, and Future Trends](https://mlsanity.com/paper/2402.15490/) (2024)
- [Exploring the Synergies of Hybrid CNNs and ViTs Architectures for Computer Vision: A survey](https://mlsanity.com/paper/2402.02941/) (2024)
- [RF-DETR Object Detection vs YOLOv12 : A Study of Transformer-based and CNN-based Architectures for Single-Class and Multi-Class Greenfruit Detection in Complex Orchard Environments Under Label Ambiguity](https://mlsanity.com/paper/2504.13099/) (2025)
- [Towards a Physics Foundation Model](https://mlsanity.com/paper/2509.13805/) (2025)
- [Local Concept Embeddings for Analysis of Concept Distributions in Vision DNN Feature Spaces](https://mlsanity.com/paper/2311.14435/) (2023)
- [DH-Mamba: Exploring Dual-domain Hierarchical State Space Models for MRI Reconstruction](https://mlsanity.com/paper/2501.08163/) (2025)
- [Improving 3D Medical Image Segmentation at Boundary Regions using Local Self-attention and Global Volume Mixing](https://mlsanity.com/paper/2410.15360/) (2024)
- [EHCTNet: Enhanced Hybrid of CNN and Transformer Network for Remote Sensing Image Change Detection](https://mlsanity.com/paper/2501.01238/) (2025)
- [AutoGen Driven Multi Agent Framework for Iterative Crime Data Analysis and Prediction](https://mlsanity.com/paper/2506.11475/) (2025)
- [Advances in LLMs with Focus on Reasoning, Adaptability, Efficiency and Ethics](https://mlsanity.com/paper/2506.12365/) (2025)
- [A Blueprint for Equilibrium-Based Differentiable Continuous-Variable Thermodynamic Computing](https://mlsanity.com/paper/2607.16183/) (2026)
- [Spatiotemporal Object Detection for Improved Aerial Vehicle Detection in Traffic Monitoring](https://mlsanity.com/paper/2410.13616/) (2024)
- [Masked Autoencoder Self Pre-Training for Defect Detection in Microelectronics](https://mlsanity.com/paper/2504.10021/) (2025)
- [Beyond Motion Cues and Structural Sparsity: Revisiting Small Moving Target Detection](https://mlsanity.com/paper/2509.07654/) (2025)
- [A Tumor Aware DenseNet Swin Hybrid Learning with Boosted and Hierarchical Feature Spaces for Large-Scale Brain MRI Classification](https://mlsanity.com/paper/2601.18330/) (2026)
- [Frequency Switching Mechanism for Parameter-E!cient Multi-Task Learning](https://mlsanity.com/paper/2603.21111/) (2026)
- [Saliency Suppressed, Semantics Surfaced: Visual Transformations in Neural Networks and the Brain](https://mlsanity.com/paper/2404.18772/) (2024)
- [Particle Multi-Axis Transformer for Jet Tagging](https://mlsanity.com/paper/2406.06638/) (2024)
- [iiANET: Inception Inspired Attention Hybrid Network for efficient Long-Range Dependency](https://mlsanity.com/paper/2407.07603/) (2024)
- [Review of Fruit Tree Image Segmentation](https://mlsanity.com/paper/2412.14631/) (2024)
- [Data-driven tool wear prediction in milling, based on a process-integrated single-sensor approach](https://mlsanity.com/paper/2412.19950/) (2024)
- [A super-resolution reconstruction method for lightweight building images based on an expanding feature modulation network](https://mlsanity.com/paper/2503.13179/) (2025)
- [A Sensorimotor Vision Transformer](https://mlsanity.com/paper/2504.02536/) (2025)
- [X-ray illicit object detection using hybrid CNN-transformer neural network architectures](https://mlsanity.com/paper/2505.00564/) (2025)
- [CascadedViT: Cascaded Chunk-FeedForward and Cascaded Group Attention Vision Transformer](https://mlsanity.com/paper/2511.14111/) (2025)
- [Implementation of a Skin Lesion Detection System for Managing Children with Atopic Dermatitis Based on Ensemble Learning](https://mlsanity.com/paper/2511.23082/) (2025)
- [Exploring Diagnostic Prompting Approach for Multimodal LLM-based Visual Complexity Assessment: A Case Study of Amazon Search Result Pages](https://mlsanity.com/paper/2512.00082/) (2025)
- [Federated Vision Transformer with Adaptive Focal Loss for Medical Image Classification](https://mlsanity.com/paper/2602.01633/) (2026)
- [Decision-Aware Attention Propagation for Vision Transformer Explainability](https://mlsanity.com/paper/2604.18094/) (2026)

## Abstract (arXiv)

Vision transformers have become popular as a possible substitute to convolutional neural networks (CNNs) for a variety of computer vision applications. These transformers, with their ability to focus on global relationships in images, offer large learning capacity. However, they may suffer from limited generalization as they do not tend to model local correlation in images. Recently, in vision transformers hybridization of both the convolution operation and self-attention mechanism has emerged, to exploit both the local and global image representations. These hybrid vision transformers, also referred to as CNN-Transformer architectures, have demonstrated remarkable results in vision applications. Given the rapidly growing number of hybrid vision transformers, it has become necessary to provide a taxonomy and explanation of these hybrid architectures. This survey presents a taxonomy of the recent vision transformer architectures and more specifically that of the hybrid vision transformers. Additionally, the key features of these architectures such as the attention mechanisms, positional embeddings, multi-scale processing, and convolution are also discussed. In contrast to the previous survey papers that are primarily focused on individual vision transformer architectures or CNNs, this survey uniquely emphasizes the emerging trend of hybrid vision transformers. By showcasing the potential of hybrid vision transformers to deliver exceptional performance across a range of computer vision tasks, this survey sheds light on the future directions of this rapidly evolving architecture.