mlsanityData to 7 Oct 2026

A Data-scalable Transformer for Medical Image Segmentation: Architecture, Model Efficiency, and Benchmark

arXiv 2203.00131 ↗First posted 28 Feb 2022eess.IV cs.CVcited by 45

Not on the map. Why?

Abstract (arXiv)

Transformers have demonstrated remarkable performance in natural language processing and computer vision. However, existing vision Transformers struggle to learn from limited medical data and are unable to generalize on diverse medical image tasks. To tackle these challenges, we present MedFormer, a data-scalable Transformer designed for generalizable 3D medical image segmentation. Our approach incorporates three key elements: a desirable inductive bias, hierarchical modeling with linear-complexity attention, and multi-scale feature fusion that integrates spatial and semantic information globally. MedFormer can learn across tiny- to large-scale data without pre-training. Comprehensive experiments demonstrate MedFormer's potential as a versatile segmentation backbone, outperforming CNNs and vision Transformers on seven public datasets covering multiple modalities (e.g., CT and MRI) and various medical targets (e.g., healthy organs, diseased tissues, and tumors). We provide public access to our models and evaluation pipeline, offering solid baselines and unbiased comparisons to advance a wide range of downstream clinical applications.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Builds on 52 of its 80 references that are in the map. First posted on arXiv 28 Feb 2022.

Builds on

52 of 80 references.

References · 52

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020Language Models are Few-Shot Learnerscited by 32,199
  6. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  7. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  8. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  9. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  10. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  11. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  12. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
38 more
  1. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  2. 2022A ConvNet for the 2020scited by 3,545
  3. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  4. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
  5. 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
  6. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  7. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  8. 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,552
  9. 2018UNet++: A Nested U-Net Architecture for Medical Image Segmentationcited by 1,518
  10. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  11. 2018Attention U-Net: Learning Where to Look for the Pancreascited by 1,420
  12. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,417
  13. 2021Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentationcited by 1,218
  14. 2021Grounded Language-Image Pre-trainingcited by 1,147
  15. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  16. 2021UNETR: Transformers for 3D Medical Image Segmentationcited by 954
  17. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  18. 2022Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Imagescited by 823
  19. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  20. 2018CCNet: Criss-Cross Attention for Semantic Segmentationcited by 717
  21. 2021The Medical Segmentation Decathloncited by 620
  22. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 566
  23. 2019The Liver Tumor Segmentation Benchmark (LiTS)cited by 504
  24. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  25. 2017Cascaded Pyramid Network for Multi-Person Pose Estimationcited by 341
  26. 2020Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentationcited by 326
  27. 2019A Multiscale Visualization of Attention in the Transformer Modelcited by 292
  28. 2018Attention Gated Networks: Learning to Leverage Salient Regions in Medical Imagescited by 271
  29. 2021CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentationcited by 227
  30. 2014Efficient Object Localization Using Convolutional Networkscited by 226
  31. 2020Synthesizer: Rethinking Self-Attention in Transformer Modelscited by 204
  32. 2019The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomescited by 168
  33. 2021RegionViT: Regional-to-Local Attention for Vision Transformerscited by 126
  34. 2021DS-TransUNet:Dual Swin Transformer U-Net for Medical Image Segmentationcited by 126
  35. 2021UTNet: A Hybrid Transformer Architecture for Medical Image Segmentationcited by 116
  36. 2016A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detectioncited by 83
  37. 2018Deep learning to achieve clinically applicable segmentation of head and neck anatomy for radiotherapycited by 76
  38. 2019FocusNet: Imbalanced Large and Small Organ Segmentation with an End-to-End Deep Neural Network for Head and Neck CT Imagescited by 24

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 10; 2024: 11; 2025: 13; 2026: 7.

45 papers.

Sort:

By area:

  1. 2024Medformer: A Multi-Granularity Patching Transformer for Medical Time-Series Classificationcited by 53
  2. 2023Training Like a Medical Resident: Context-Prior Learning Toward Universal Medical Image Segmentationcited by 32
  3. 2024Touchstone Benchmark: Are We on the Right Way for Evaluating AI Algorithms for Medical Segmentation?cited by 31
  4. 2024Data-Centric Foundation Models in Computational Healthcare: A Surveycited by 27
  5. 2023From Google Gemini to OpenAI Q* (Q-Star): A Survey of Reshaping the Generative Artificial Intelligence (AI) Research Landscapecited by 18
  6. 2022DeepRecon: Joint 2D Cardiac Segmentation and 3D Volume Reconstruction via A Structure-Specific Generative Methodcited by 17
  7. 2023E2ENet: Dynamic Sparse Feature Fusion for Accurate and Efficient 3D Medical Image Segmentationcited by 16
  8. 2025Show and Segment: Universal Medical Image Segmentation via In-Context Learningcited by 15
  9. 2024Layout Agnostic Scene Text Image Synthesis with Diffusion Modelscited by 14
  10. 2022Region Proposal Rectification Towards Robust Instance Segmentation of Biological Imagescited by 13
  11. 2023DeFormer: Integrating Transformers with Deformable Models for 3D Shape Abstraction from a Single Imagecited by 11
  12. 2024BEFUnet: A Hybrid CNN-Transformer Architecture for Precise Medical Image Segmentationcited by 11
33 more
  1. 2025Learning Segmentation from Radiology Reportscited by 11
  2. 2023DMCVR: Morphology-Guided Diffusion Model for 3D Cardiac Volume Reconstructioncited by 10
  3. 2025Diffusion-empowered AutoPrompt MedSAMcited by 10
  4. 2022The Lighter The Better: Rethinking Transformers in Medical Image Segmentation Through Adaptive Pruningcited by 9
  5. 2022Vision Transformers in Medical Imaging: A Reviewcited by 8
  6. 2025Scaling Artificial Intelligence for Multi-Tumor Early Detection with More Reports, Fewer Maskscited by 8
  7. 2025LUCAS: Layered Universal Codec Avatarscited by 7
  8. 2023SA2-Net: Scale-aware Attention Network for Microscopic Image Segmentationcited by 6
  9. 2023Deep Deformable Models: Learning 3D Shape Abstractions with Part Consistencycited by 5
  10. 2025FluenceFormer: Transformer-Driven Multi-Beam Fluence Map Regression for Radiotherapy Planningcited by 4
  11. 2023AViT: Adapting Vision Transformers for Small Skin Lesion Segmentation Datasetscited by 3
  12. 2024Inter- and intra-uncertainty based feature aggregation model for semi-supervised histopathology image segmentationcited by 3
  13. 2024Instantaneous Perception of Moving Objects in 3Dcited by 3
  14. 2025See More, Change Less: Anatomy-Aware Diffusion for Contrast Enhancementcited by 3
  15. 2023ADS_UNet: A Nested UNet for Histopathology Image Segmentationcited by 2
  16. 2024Continuous Spatio-Temporal Memory Networks for 4D Cardiac Cine MRI Segmentationcited by 2
  17. 2024S3-Mamba: Small-Size-Sensitive Mamba for Lesion Segmentationcited by 2
  18. 2024VerSe: Integrating Multiple Queries as Prompts for Versatile Cardiac MRI Segmentationcited by 2
  19. 2025Latent Interpolation Learning Using Diffusion Models for Cardiac Volume Reconstructioncited by 2
  20. 2025Auditing Significance, Metric Choice, and Demographic Fairness in Medical AI Challengescited by 2
  21. 2025BiPVL-Seg: Bidirectional Progressive Vision-Language Fusion with Global-Local Alignment for Medical Image Segmentationcited by 1
  22. 2026GLeVE: Graph-Guided Lesion Grounding with Proposal Verification in 3D CTcited by 1
  23. 2023Image-level supervision and self-training for transformer-based cross-modality tumor segmentationcited by 0
  24. 2024FAN-Unet: Enhancing Unet with vision Fourier Analysis Block for Biomedical Image Segmentationcited by 0
  25. 2025Unified Start, Personalized End: Progressive Pruning for Efficient 3D Medical Image Segmentationcited by 0
  26. 2025Towards Data-Efficient Medical Imaging: A Generative and Semi-Supervised Frameworkcited by 0
  27. 2025Bridging spatial awareness and global context in medical image segmentationcited by 0
  28. 2026CDSA-Net:Collaborative Decoupling of Vascular Structure and Background for High-Fidelity Coronary Digital Subtraction Angiographycited by 0
  29. 2026Deep Learning-Based Automated Quantification of TIMI Myocardial Perfusion Frame Count (DL-TMPFC) from Coronary Angiography: A Novel Framework for Rapid Assessment of Microvascular Dysfunctioncited by 0
  30. 2026MedPlex: Deep Vision-Language Co-Adaptation for Clinically Grounded Medical Segmentationcited by 0
  31. 2026Report Supervisioncited by 0
  32. 2026RT-Super: Learning Tumor Segmentation from Longitudinal Images and Reportscited by 0
  33. 2026Merlin Plus: A Large-Scale, Multi-Cancer, Image-Mask-Report Datasetcited by 0