# Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives

arXiv: https://arxiv.org/abs/2206.01136 · HTML: https://mlsanity.com/paper/2206.01136/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 239 of its 435 references that are in the map.
- First posted on arXiv 2 Jun 2022.

## Builds on (most cited first, 30 of 239)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)

## Built on by (most cited first, 30 of 33)

- [NeuroGraph: Benchmarks for Graph Machine Learning in Brain Connectomics](https://mlsanity.com/paper/2306.06202/) (2023)
- [A Recent Survey of Vision Transformers for Medical Image Segmentation](https://mlsanity.com/paper/2312.00634/) (2023)
- [DenoMamba: A fused state-space model for low-dose CT denoising](https://mlsanity.com/paper/2409.13094/) (2024)
- [Which Transformer to Favor: A Comparative Analysis of Efficiency in Vision Transformers](https://mlsanity.com/paper/2308.09372/) (2023)
- [Recent Advances in Medical Imaging Segmentation: A Survey](https://mlsanity.com/paper/2505.09274/) (2025)
- [Efficiently Training Vision Transformers on Structural MRI Scans for Alzheimer's Disease Detection](https://mlsanity.com/paper/2303.08216/) (2023)
- [Revisiting Computer-Aided Tuberculosis Diagnosis](https://mlsanity.com/paper/2307.02848/) (2023)
- [Data efficient deep learning for medical image analysis: A survey](https://mlsanity.com/paper/2310.06557/) (2023)
- [Transformers-based architectures for stroke segmentation: A review](https://mlsanity.com/paper/2403.18637/) (2024)
- [MedAugment: Universal Automatic Data Augmentation Plug-in for Medical Image Analysis](https://mlsanity.com/paper/2306.17466/) (2023)
- [Learning Low-Rank Feature for Thorax Disease Classification](https://mlsanity.com/paper/2404.18933/) (2024)
- [BreastDCEDL: A Comprehensive Breast Cancer DCE-MRI Dataset and Transformer Implementation for Treatment Response Prediction](https://mlsanity.com/paper/2506.12190/) (2025)
- [MIAFEx: An Attention-based Feature Extraction Method for Medical Image Classification](https://mlsanity.com/paper/2501.08562/) (2025)
- [LoSA-Net: A Localized and Scale-Adaptive Network for Boundary-Sensitive Prediction of Perineural Invasion in 3D MRI](https://mlsanity.com/paper/2607.10992/) (2026)
- [UNesT: Local Spatial Representation Learning with Hierarchical Transformer for Efficient Medical Segmentation](https://mlsanity.com/paper/2209.14378/) (2022)
- [Vision Transformers and Bi-LSTM for Alzheimer's Disease Diagnosis from 3D MRI](https://mlsanity.com/paper/2401.03132/) (2024)
- [Evaluating transfer learning strategies for improving dairy cattle body weight prediction in small farms using depth-image and point-cloud data](https://mlsanity.com/paper/2601.01044/) (2026)
- [JANUS: Anatomy-Conditioned Gating for Robust CT Triage Under Distribution Shift](https://mlsanity.com/paper/2605.13813/) (2026)
- [Developing a Novel Image Marker to Predict the Clinical Outcome of Neoadjuvant Chemotherapy (NACT) for Ovarian Cancer Patients](https://mlsanity.com/paper/2309.07087/) (2023)
- [MVC: A Multi-Task Vision Transformer Network for COVID-19 Diagnosis from Chest X-ray Images](https://mlsanity.com/paper/2310.00418/) (2023)
- [Shape Modeling of Longitudinal Medical Images: From Diffeomorphic Metric Mapping to Deep Learning](https://mlsanity.com/paper/2503.21489/) (2025)
- [Toward Next-generation Medical Vision Backbones: Modeling Finer-grained Long-range Visual Dependency](https://mlsanity.com/paper/2509.11328/) (2025)
- [CoMViT: An Efficient Vision Backbone for Supervised Classification in Medical Imaging](https://mlsanity.com/paper/2510.27442/) (2025)
- [Validating Deep Models for Alzheimer's 18F-FDG PET Diagnosis Across Populations: A Study with Latin American Data](https://mlsanity.com/paper/2511.00728/) (2025)
- [High-Resolution Magnetic Particle Imaging System Matrix Recovery Using a Vision Transformer with Residual Feature Network](https://mlsanity.com/paper/2511.02212/) (2025)
- [BertsWin: Resolving Topological Sparsity in 3D Masked Autoencoders via Component-Balanced Structural Optimization](https://mlsanity.com/paper/2512.21769/) (2025)
- [Segmenting Low-Contrast XCTs of Concrete: An Unsupervised Approach](https://mlsanity.com/paper/2603.00127/) (2026)
- [Neural Dynamics-Informed Pre-trained Framework for Personalized Brain Functional Network Construction](https://mlsanity.com/paper/2603.07524/) (2026)
- [Modeling Local, Global, and Cross-Modal Context in Multimodal 3D MRI](https://mlsanity.com/paper/2606.26894/) (2026)
- [MMA-Former: Multi-Window Mixture-of-Head Attention Transformer for Adaptive PNI Prediction in 3D MRI](https://mlsanity.com/paper/2607.10988/) (2026)

## Abstract (arXiv)

Transformer, the latest technological advance of deep learning, has gained prevalence in natural language processing or computer vision. Since medical imaging bear some resemblance to computer vision, it is natural to inquire about the status quo of Transformers in medical imaging and ask the question: can the Transformer models transform medical imaging? In this paper, we attempt to make a response to the inquiry. After a brief introduction of the fundamentals of Transformers, especially in comparison with convolutional neural networks (CNNs), and highlighting key defining properties that characterize the Transformers, we offer a comprehensive review of the state-of-the-art Transformer-based approaches for medical imaging and exhibit current research progresses made in the areas of medical image segmentation, recognition, detection, registration, reconstruction, enhancement, etc. In particular, what distinguishes our review lies in its organization based on the Transformer's key defining properties, which are mostly derived from comparing the Transformer and CNN, and its type of architecture, which specifies the manner in which the Transformer and CNN are combined, all helping the readers to best understand the rationale behind the reviewed approaches. We conclude with discussions of future perspectives.