mlsanityData to 7 Oct 2026

A Recent Survey of Vision Transformers for Medical Image Segmentation

arXiv 2312.00634 ↗First posted 1 Dec 2023eess.IV cs.CVcited by 22

Not on the map. Why?

Abstract (arXiv)

Medical image segmentation plays a crucial role in various healthcare applications, enabling accurate diagnosis, treatment planning, and disease monitoring. Traditionally, convolutional neural networks (CNNs) dominated this domain, excelling at local feature extraction. However, their limitations in capturing long-range dependencies across image regions pose challenges for segmenting complex, interconnected structures often encountered in medical data. In recent years, Vision Transformers (ViTs) have emerged as a promising technique for addressing the challenges in medical image segmentation. Their multi-scale attention mechanism enables effective modeling of long-range dependencies between distant structures, crucial for segmenting organs or lesions spanning the image. Additionally, ViTs' ability to discern subtle pattern heterogeneity allows for the precise delineation of intricate boundaries and edges, a critical aspect of accurate medical image segmentation. However, they do lack image-related inductive bias and translational invariance, potentially impacting their performance. Recently, researchers have come up with various ViT-based approaches that incorporate CNNs in their architectures, known as Hybrid Vision Transformers (HVTs) to capture local correlation in addition to the global information in the images. This survey paper provides a detailed review of the recent advancements in ViTs and HVTs for medical image segmentation. Along with the categorization of ViT and HVT-based medical image segmentation approaches, we also present a detailed overview of their real-time applications in several medical image modalities. This survey may serve as a valuable resource for researchers, healthcare practitioners, and students in understanding the state-of-the-art approaches for ViT-based medical image segmentation.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 74 of its 186 references that are in the map. First posted on arXiv 1 Dec 2023.

Builds on

74 of 186 references.

References · 74

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,552
  6. 2021Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentationcited by 1,218
  7. 2017A Survey on Deep Learning in Medical Image Analysiscited by 1,182
  8. 2021Transformers in Vision: A Surveycited by 1,009
  9. 2021UNETR: Transformers for 3D Medical Image Segmentationcited by 954
  10. 2022Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Imagescited by 823
  11. 2020A Survey on Visual Transformercited by 653
  12. 2021The Medical Segmentation Decathloncited by 620
38 more
  1. 2022MaxViT: Multi-Axis Vision Transformercited by 476
  2. 2018HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Imagescited by 387
  3. 2021CMT: Convolutional Neural Networks Meet Vision Transformerscited by 310
  4. 2022Diffusion Models for Medical Image Analysis: A Comprehensive Surveycited by 269
  5. 2021TransBTS: Multimodal Brain Tumor Segmentation Using Transformercited by 265
  6. 2021TransFuse: Fusing Transformers and CNNs for Medical Image Segmentationcited by 258
  7. 2022Transformers in Medical Imaging: A Surveycited by 245
  8. 2021CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentationcited by 227
  9. 2019A Survey of the Recent Architectures of Deep Convolutional Neural Networkscited by 211
  10. 2021nnFormer: Interleaved Transformer for Volumetric Segmentationcited by 179
  11. 2019The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomescited by 168
  12. 2020Medical Image Segmentation Using Deep Learning: A Surveycited by 163
  13. 2019Deep Semantic Segmentation of Natural and Medical Images: A Reviewcited by 124
  14. 2021UTNet: A Hybrid Transformer Architecture for Medical Image Segmentationcited by 116
  15. 2022ReSTR: Convolution-free Referring Image Segmentation Using Transformerscited by 112
  16. 2021ViTGAN: Training GANs with Vision Transformerscited by 110
  17. 2023CellViT: Vision Transformers for Precise Cell Segmentation and Classificationcited by 108
  18. 2017Deep Learning for Medical Image Processing: Overview, Challenges and Futurecited by 104
  19. 2017Medical Image Analysis using Convolutional Neural Networks: A Reviewcited by 99
  20. 2018MILD-Net: Minimal Information Loss Dilated Network for Gland Instance Segmentation in Colon Histology Imagescited by 87
  21. 2021ViT-V-Net: Vision Transformer for Unsupervised Volumetric Medical Image Registrationcited by 87
  22. 2021TransMed: Transformers Advance Multi-modal Medical Image Classificationcited by 85
  23. 2021Deep Learning for Chest X-ray Analysis: A Surveycited by 84
  24. 2021BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Searchcited by 63
  25. 2023Advances in Medical Image Analysis with Vision Transformers: A Comprehensive Reviewcited by 62
  26. 2021Medical Image Segmentation Using Squeeze-and-Expansion Transformerscited by 56
  27. 2021AFTer-UNet: Axial Fusion Transformer UNet for Medical Image Segmentationcited by 48
  28. 2020NuClick: A Deep Learning Framework for Interactive Segmentation of Microscopy Imagescited by 46
  29. 2021DPT: Deformable Patch-based Transformer for Visual Recognitioncited by 43
  30. 2022TransDeepLab: Convolution-Free Transformer-based DeepLab v3+ for Medical Image Segmentationcited by 43
  31. 2021A Robust Volumetric Transformer for Accurate 3D Tumor Segmentationcited by 41
  32. 2018Recent progress in semantic image segmentationcited by 39
  33. 2021Boundary-aware Transformers for Skin Lesion Segmentationcited by 39
  34. 2023A survey of the Vision Transformers and their CNN-Transformer based Variantscited by 33
  35. 2022Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectivescited by 32
  36. 2023A comprehensive survey on deep active learning in medical image analysiscited by 32
  37. 2022ColonFormer: An Efficient Transformer based Method for Colon Polyp Segmentationcited by 28
  38. 2023A Systematic Review of Few-Shot Learning in Medical Imagingcited by 28

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 16; 2026: 3.

22 papers.

Sort:

By area:

  1. 2025Spiking Transformer:Introducing Accurate Addition-Only Spiking Self-Attention for Transformercited by 13
  2. 2025Fit Pixels, Get Labels: Meta-learned Implicit Networks for Image Segmentationcited by 11
  3. 2025A Survey on Self-supervised Contrastive Learning for Multimodal Text-Image Analysiscited by 5
  4. 2025MedNeXt-v2: Scaling 3D ConvNeXts for Large-Scale Supervised Representation Learning in Medical Image Segmentationcited by 5
  5. 2024RS-FME-SwinT: A Novel Feature Map Enhancement Framework Integrating Customized SwinT with Residual and Spatial CNN for Monkeypox Diagnosiscited by 4
  6. 2025A Novel Channel Boosted Residual CNN-Transformer with Regional-Boundary Learning for Breast Cancer Detectioncited by 3
  7. 2025CE-RS-SBCIT A Novel Channel Enhanced Hybrid CNN Transformer with Residual, Spatial, and Boundary-Aware Learning for Brain Tumor MRI Analysiscited by 3
  8. 2025RS-CA-HSICT: A Residual and Spatial Channel Augmented CNN Transformer Framework for Monkeypox Detectioncited by 3
  9. 2025AutoGen Driven Multi Agent Framework for Iterative Crime Data Analysis and Predictioncited by 2
  10. 2025Residual-SwinCA-Net: A Channel-Aware Integrated Residual CNN-Swin Transformer for Malignant Lesion Segmentation in BUSIcited by 2
  11. 2025Evaluation of Vision Transformers for Multimodal Image Classification: A Case Study on Brain, Lung, and Kidney Tumorscited by 1
  12. 2025Advanced Hybrid Transformer LSTM Technique with Attention and TS Mixer for Drilling Rate of Penetration Predictioncited by 1
10 more
  1. 2025VL-OrdinalFormer: Vision Language Guided Ordinal Transformers for Interpretable Knee Osteoarthritis Gradingcited by 1
  2. 2026RSwinV2-MD: An Enhanced Residual SwinV2 Transformer for Monkeypox Detection from Skin Imagescited by 1
  3. 2026T-DuMpRa: Teacher-guided Dual-path Multi-prototype Retrieval Augmented framework for fine-grained medical image classificationcited by 1
  4. 2024A Novel Decision Ensemble Framework: Customized Attention-BiLSTM and XGBoost for Speculative Stock Price Forecastingcited by 0
  5. 2024Particle Multi-Axis Transformer for Jet Taggingcited by 0
  6. 2025VerteNet -- A Multi-Context Hybrid CNN Transformer for Accurate Vertebral Landmark Localization in Lateral Spine DXA Imagescited by 0
  7. 2025Automated MRI Tumor Segmentation using hybrid U-Net with Transformer and Efficient Attentioncited by 0
  8. 2025Are Vision Transformer Representations Semantically Meaningful? A Case Study in Medical Imagingcited by 0
  9. 2025Attention via Synaptic Plasticity is All You Need: A Biologically Inspired Spiking Neuromorphic Transformercited by 0
  10. 2026Token-UNet: A New Case for Transformers Integration in Efficient and Interpretable 3D UNets for Brain Imaging Segmentationcited by 0