mlsanityData to 7 Oct 2026

UNETR: Transformers for 3D Medical Image Segmentation

arXiv 2103.10504 ↗First posted 18 Mar 2021eess.IV cs.CV cs.LGcited by 954

Not on the map. Why?

Cited by 56 in State-space sequence models (S4, Mamba), but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Fully Convolutional Neural Networks (FCNNs) with contracting and expanding paths have shown prominence for the majority of medical image segmentation applications since the past decade. In FCNNs, the encoder plays an integral role by learning both global and local features and contextual representations which can be utilized for semantic output prediction by the decoder. Despite their success, the locality of convolutional layers in FCNNs, limits the capability of learning long-range spatial dependencies. Inspired by the recent success of transformers for Natural Language Processing (NLP) in long-range sequence learning, we reformulate the task of volumetric (3D) medical image segmentation as a sequence-to-sequence prediction problem. We introduce a novel architecture, dubbed as UNEt TRansformers (UNETR), that utilizes a transformer as the encoder to learn sequence representations of the input volume and effectively capture the global multi-scale information, while also following the successful "U-shaped" network design for the encoder and decoder. The transformer encoder is directly connected to a decoder via skip connections at different resolutions to compute the final semantic segmentation output. We have validated the performance of our method on the Multi Atlas Labeling Beyond The Cranial Vault (BTCV) dataset for multi-organ segmentation and the Medical Segmentation Decathlon (MSD) dataset for brain tumor and spleen segmentation tasks. Our benchmarks demonstrate new state-of-the-art performance on the BTCV leaderboard. Code: https://monai.io/research/unetr

Key facts. Cited by 954 papers in the map (data through 7 Oct 2026). Builds on 43 of its 59 references that are in the map. First posted on arXiv 18 Mar 2021.

Builds on

43 of 59 references.

References · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2016Layer Normalizationcited by 5,579
  7. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  8. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  9. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  10. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  11. 2017Non-local Neural Networkscited by 2,290
  12. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
31 more
  1. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  2. 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,552
  3. 2018UNet++: A Nested U-Net Architecture for Medical Image Segmentationcited by 1,518
  4. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,471
  5. 2018Attention U-Net: Learning Where to Look for the Pancreascited by 1,420
  6. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  7. 2018Dual Attention Network for Scene Segmentationcited by 1,001
  8. 2018Self-Attention Generative Adversarial Networkscited by 809
  9. 2019Stand-Alone Self-Attention in Vision Modelscited by 542
  10. 2018Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challengecited by 508
  11. 2019A large annotated medical image dataset for the development and evaluation of segmentation algorithmscited by 371
  12. 20183D MRI brain tumor segmentation using autoencoder regularizationcited by 324
  13. 2021Medical Transformer: Gated Axial-Attention for Medical Image Segmentationcited by 319
  14. 2021TransBTS: Multimodal Brain Tumor Segmentation Using Transformercited by 265
  15. 2016Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentationcited by 263
  16. 2021TransFuse: Fusing Transformers and CNNs for Medical Image Segmentationcited by 258
  17. 2021CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentationcited by 227
  18. 2019CE-Net: Context Encoder Network for 2D Medical Image Segmentationcited by 219
  19. 2019Local Relation Networks for Image Recognitioncited by 210
  20. 2019The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomescited by 168
  21. 2021LambdaNetworks: Modeling Long-Range Interactions Without Attentioncited by 107
  22. 2019Prior-aware Neural Network for Partially-Supervised Multi-Organ Segmentationcited by 88
  23. 20183D Semi-Supervised Learning with Uncertainty-Aware Multi-View Co-Trainingcited by 57
  24. 20163D Deeply Supervised Network for Automatic Liver Segmentation from CT Volumescited by 43
  25. 2018RA-UNet: A hybrid deep attention-aware network to extract liver and tumor in CT scanscited by 41
  26. 20173D Anisotropic Hybrid Network: Transferring Convolutional Features from 2D Images to 3D Anisotropic Volumescited by 38
  27. 2017On the Compactness, Efficiency, and Representation of 3D Convolutional Networks: Brain Parcellation as a Pretext Taskcited by 36
  28. 2017Deeply-Supervised CNN for Prostate Segmentationcited by 30
  29. 2016Combining Fully Convolutional and Recurrent Neural Networks for 3D Biomedical Image Segmentationcited by 25
  30. 2019An attempt at beating the 3D U-Netcited by 21
  31. 2017Hierarchical 3D fully convolutional networks for multi-organ segmentationcited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 22; 2022: 138; 2023: 171; 2024: 237; 2025: 225; 2026: 161.

954 papers.

Sort:

By area:

  1. 2021Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentationcited by 1,218
  2. 2022Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Imagescited by 823
  3. 2023Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentationcited by 485
  4. 2021Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysiscited by 418
  5. 2024U-Mamba: Enhancing Long-range Dependency for Biomedical Image Segmentationcited by 412
  6. 2022AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentationcited by 347
  7. 2024SegMamba: Long-range Sequential Modeling Mamba For 3D Medical Image Segmentationcited by 266
  8. 2022Transformers in Medical Imaging: A Surveycited by 245
  9. 2024nnU-Net Revisited: A Call for Rigorous Validation in 3D Medical Image Segmentationcited by 235
  10. 2023CLIP-Driven Universal Model for Organ Segmentation and Tumor Detectioncited by 224
  11. 2021UCTransNet: Rethinking the Skip Connections in U-Net from a Channel-wise Perspective with Transformercited by 197
  12. 2022Medical Image Segmentation Review: The success of U-Netcited by 190
38 more
  1. 2022UNeXt: MLP-based Rapid Medical Image Segmentation Networkcited by 185
  2. 2021nnFormer: Interleaved Transformer for Volumetric Segmentationcited by 179
  3. 2024Swin-UMamba: Mamba-based UNet with ImageNet-based pretrainingcited by 179
  4. 2024U-KAN Makes Strong Backbone for Medical Image Segmentation and Generationcited by 176
  5. 2023MedNeXt: Transformer-driven Scaling of ConvNets for Medical Image Segmentationcited by 161
  6. 2024Medical SAM 2: Segment medical images as video via Segment Anything Model 2cited by 151
  7. 2024Mamba-UNet: UNet-Like Pure Visual Mamba for Medical Image Segmentationcited by 133
  8. 2023MedSegDiff-V2: Diffusion based Medical Image Segmentation with Transformercited by 124
  9. 2023Transformer-Based Visual Segmentation: A Surveycited by 124
  10. 2022UNETR++: Delving into Efficient and Accurate 3D Medical Image Segmentationcited by 112
  11. 2023CellViT: Vision Transformers for Precise Cell Segmentation and Classificationcited by 108
  12. 2024BiomedParse: a biomedical foundation model for image parsing of everything everywhere all at oncecited by 107
  13. 2021Recent advances and clinical applications of deep learning in medical image analysiscited by 104
  14. 2021MISSFormer: An Effective Medical Image Segmentation Transformercited by 103
  15. 2023SAM-Med3D: Towards General-purpose Segmentation Models for Volumetric Medical Imagescited by 101
  16. 2021Semi-Supervised Medical Image Segmentation via Cross Teaching between CNN and Transformercited by 99
  17. 2023STU-Net: Scalable and Transferable Medical Image Segmentation Models Empowered by Large-Scale Supervised Pre-trainingcited by 96
  18. 2022mmFormer: Multimodal Medical Transformer for Incomplete Multimodal Learning of Brain Tumor Segmentationcited by 94
  19. 2022HiFormer: Hierarchical Multi-scale Representations Using Transformers for Medical Image Segmentationcited by 94
  20. 20223D UX-Net: A Large Kernel Volumetric ConvNet Modernizing Hierarchical Transformer for Medical Image Segmentationcited by 94
  21. 2023SegVol: Universal and Interactive Volumetric Medical Image Segmentationcited by 86
  22. 2024Mamba-ND: Selective State Space Modeling for Multi-Dimensional Datacited by 84
  23. 2022Masked Image Modeling Advances 3D Medical Image Analysiscited by 71
  24. 2022Self Pre-training with Masked Autoencoders for Medical Image Classification and Segmentationcited by 69
  25. 2022Delving into Masked Autoencoders for Multi-Label Thorax Disease Classificationcited by 65
  26. 2021Are we ready for a new paradigm shift? A Survey on Visual Deep MLPcited by 63
  27. 2022Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysiscited by 62
  28. 2024VISTA3D: A Unified Segmentation Foundation Model For 3D Medical Imagingcited by 62
  29. 2023Large AI Models in Health Informatics: Applications, Challenges, and the Futurecited by 61
  30. 2023LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matchingcited by 60
  31. 2023Large-Vocabulary Segmentation for Medical Images with Text Promptscited by 60
  32. 2023Label-Free Liver Tumor Segmentationcited by 56
  33. 2021UniMiSS: Universal Medical Self-Supervised Learning via Breaking Dimensionality Barriercited by 55
  34. 2022ImageCAS: A Large-Scale Dataset and Benchmark for Coronary Artery Segmentation based on Computed Tomography Angiography Imagescited by 54
  35. 2024LightM-UNet: Mamba Assists in Lightweight UNet for Medical Image Segmentationcited by 54
  36. 2023AbdomenAtlas-8K: Annotating 8,000 CT Volumes for Multi-Organ Segmentation in Three Weekscited by 52
  37. 2024Vivim: a Video Vision Mamba for Medical Video Segmentationcited by 51
  38. 2024Vision-LSTM: xLSTM as Generic Vision Backbonecited by 51

50 most cited of 954 All 954 (JSON)