mlsanityData to 7 Oct 2026

U-Net Transformer: Self and Cross Attention for Medical Image Segmentation

arXiv 2103.06104 ↗First posted 10 Mar 2021eess.IV cs.CVcited by 81

Not on the map. Why?

Abstract (arXiv)

Medical image segmentation remains particularly challenging for complex and low-contrast anatomical structures. In this paper, we introduce the U-Transformer network, which combines a U-shaped architecture for image segmentation with self- and cross-attention from Transformers. U-Transformer overcomes the inability of U-Nets to model long-range contextual interactions and spatial dependencies, which are arguably crucial for accurate segmentation in challenging contexts. To this end, attention mechanisms are incorporated at two main levels: a self-attention module leverages global interactions between encoder features, while cross-attention in the skip connections allows a fine spatial recovery in the U-Net decoder by filtering out non-semantic features. Experiments on two abdominal CT-image datasets show the large performance gain brought out by U-Transformer compared to U-Net and local Attention U-Nets. We also highlight the importance of using both self- and cross-attention, and the nice interpretability features brought out by U-Transformer.

Key facts. Cited by 81 papers in the map (data through 7 Oct 2026). Builds on 18 of its 21 references that are in the map. First posted on arXiv 10 Mar 2021.

Builds on

18 of 21 references.

References · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2017Non-local Neural Networkscited by 2,290
  7. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  8. 2018UNet++: A Nested U-Net Architecture for Medical Image Segmentationcited by 1,518
  9. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,471
  10. 2021Transformers in Vision: A Surveycited by 1,009
  11. 2018Dual Attention Network for Scene Segmentationcited by 1,001
  12. 2021Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attentioncited by 468
6 more
  1. 2020Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentationcited by 326
  2. 2018Attention Gated Networks: Learning to Leverage Salient Regions in Medical Imagescited by 271
  3. 2019Cross-Modal Self-Attention Network for Referring Image Segmentationcited by 263
  4. 2018Concurrent Spatial and Channel Squeeze & Excitation in Fully Convolutional Networkscited by 120
  5. 2019Multi-scale self-guided attention for medical image segmentationcited by 84
  6. 2020Real-time Semantic Segmentation with Fast Attentioncited by 36

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 19; 2023: 22; 2024: 19; 2025: 15; 2026: 3.

81 papers.

Sort:

By area:

  1. 2022MAXIM: Multi-Axis MLP for Image Processingcited by 301
  2. 2023DiffIR: Efficient Diffusion Model for Image Restorationcited by 253
  3. 2022Transformers in Medical Imaging: A Surveycited by 245
  4. 2022Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imagingcited by 66
  5. 2024SafeGen: Mitigating Sexually Explicit Content Generation in Text-to-Image Modelscited by 64
  6. 2023GridFormer: Residual Dense Transformer with Grid Structure for Image Restoration in Adverse Weather Conditionscited by 62
  7. 2025HVI: A New Color Space for Low-light Image Enhancementcited by 62
  8. 2022Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectivescited by 32
  9. 2022The Fully Convolutional Transformer for Medical Image Segmentationcited by 31
  10. 2021More than Encoder: Introducing Transformer Decoder to Upsamplecited by 26
  11. 2024You Only Need One Color Space: An Efficient Network for Low-light Image Enhancementcited by 24
  12. 2022Vision Transformers in Medical Computer Vision -- A Contemplative Retrospectioncited by 22
38 more
  1. 2022Transformer Meets Boundary Value Inverse Problemscited by 22
  2. 2025Primus: Enforcing Attention Usage for 3D Medical Image Segmentationcited by 22
  3. 2022On Optimizing the Communication of Model Parallelismcited by 19
  4. 2024ExtremeCast: Boosting Extreme Value Prediction for Global Weather Forecastcited by 19
  5. 2023Transformers in Healthcare: A Surveycited by 17
  6. 2022Few-Shot Learning Meets Transformer: Unified Query-Support Transformers for Few-Shot Classificationcited by 16
  7. 2022CAT-Net: A Cross-Slice Attention Transformer Model for Prostate Zonal Segmentation in MRIcited by 15
  8. 2022A survey on attention mechanisms for medical applications: are we moving towards better algorithms?cited by 15
  9. 2023ASCON: Anatomy-aware Supervised Contrastive Learning Framework for Low-dose CT Denoisingcited by 15
  10. 2022Speech Denoising in the Waveform Domain with Self-Attentioncited by 13
  11. 2023Optimization-Inspired Cross-Attention Transformer for Compressive Sensingcited by 12
  12. 2023MIST: Medical Image Segmentation Transformer with Convolutional Attention Mixing (CAM) Decodercited by 12
  13. 2021Yformer: U-Net Inspired Transformer Architecture for Far Horizon Time Series Forecastingcited by 11
  14. 2023Transformers in Reinforcement Learning: A Surveycited by 11
  15. 2022Do we really need temporal convolutions in action segmentation?cited by 10
  16. 2022Clinically-Inspired Multi-Agent Transformers for Disease Trajectory Forecasting from Multimodal Datacited by 10
  17. 2022Patcher: Patch Transformers with Mixture of Experts for Precise Medical Image Segmentationcited by 9
  18. 2023Attention Mechanisms in Medical Image Segmentation: A Surveycited by 9
  19. 2022High-Resolution Swin Transformer for Automatic Medical Image Segmentationcited by 8
  20. 2023Dilated-UNet: A Fast and Accurate Medical Image Segmentation Approach using a Dilated Transformer and U-Net Architecturecited by 8
  21. 2023U-MixFormer: UNet-like Transformer with Mix-Attention for Efficient Semantic Segmentationcited by 8
  22. 2024Exploring the design space of deep-learning-based weather forecasting systemscited by 8
  23. 2023CSAM: A 2.5D Cross-Slice Attention Module for Anisotropic Volumetric Medical Image Segmentationcited by 7
  24. 2024Adapting Vision Foundation Models for Robust Cloud Segmentation in Remote Sensing Imagescited by 7
  25. 2023TUNeS: A Temporal U-Net with Self-Attention for Video-based Surgical Phase Recognitioncited by 6
  26. 2024Trans2Unet: Neural fusion for Nuclei Semantic Segmentationcited by 5
  27. 2023EFormer: Enhanced Transformer towards Semantic-Contour Features of Foreground for Portraits Mattingcited by 4
  28. 2023Interpretable Medical Imagery Diagnosis with Self-Attentive Transformers: A Review of Explainable AI for Health Carecited by 4
  29. 2024WaveletFormerNet: A Transformer-based Wavelet Network for Real-world Non-homogeneous and Dense Fog Removalcited by 4
  30. 2023Exploiting Partial Common Information Microstructure for Multi-Modal Brain Tumor Segmentationcited by 3
  31. 2024Rethinking Attention Gated with Hybrid Dual Pyramid Transformer-CNN for Generalized Segmentation in Medical Imagingcited by 3
  32. 20223D Shuffle-Mixer: An Efficient Context-Aware Vision Learner of Transformer-MLP Paradigm for Dense Prediction in Medical Volumecited by 2
  33. 2023D-TrAttUnet: Dual-Decoder Transformer-Based Attention Unet Architecture for Binary and Multi-classes Covid-19 Infection Segmentationcited by 2
  34. 2024D-TrAttUnet: Toward Hybrid CNN-Transformer Architecture for Generic and Subtle Segmentation in Medical Imagescited by 2
  35. 2024ECAFormer: Low-light Image Enhancement using Cross Attentioncited by 2
  36. 2025Large EEG-U-Transformer for Time-Step Level Detection Without Pre-Trainingcited by 2
  37. 2022DXM-TransFuse U-net: Dual Cross-Modal Transformer Fusion U-net for Automated Nerve Identificationcited by 1
  38. 2023PLU-Net: Extraction of multi-scale feature fusioncited by 1

50 most cited of 81 All 81 (JSON)