# D-Former: A U-shaped Dilated Transformer for 3D Medical Image Segmentation

arXiv: https://arxiv.org/abs/2201.00462 · HTML: https://mlsanity.com/paper/2201.00462/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 62 of its 84 references that are in the map.
- First posted on arXiv 3 Jan 2022.

## Builds on (most cited first, 30 of 62)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Transformer in Transformer](https://mlsanity.com/paper/2103.00112/) (2021)
- [Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs](https://mlsanity.com/paper/1412.7062/) (2014)

## Built on by (most cited first, 30 of 35)

- [nnU-Net Revisited: A Call for Rigorous Validation in 3D Medical Image Segmentation](https://mlsanity.com/paper/2404.09556/) (2024)
- [Medical Image Segmentation Review: The success of U-Net](https://mlsanity.com/paper/2211.14830/) (2022)
- [Transformers in Medical Image Analysis: A Review](https://mlsanity.com/paper/2202.12165/) (2022)
- [TransNorm: Transformer Provides a Strong Spatial Normalization Mechanism for a Deep Segmentation Model](https://mlsanity.com/paper/2207.13415/) (2022)
- [LKM-UNet: Large Kernel Vision Mamba UNet for Medical Image Segmentation](https://mlsanity.com/paper/2403.07332/) (2024)
- [Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives](https://mlsanity.com/paper/2206.01136/) (2022)
- [The Fully Convolutional Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2206.00566/) (2022)
- [HUMUS-Net: Hybrid unrolled multi-scale network architecture for accelerated MRI reconstruction](https://mlsanity.com/paper/2203.08213/) (2022)
- [Beyond Self-Attention: Deformable Large Kernel Attention for Medical Image Segmentation](https://mlsanity.com/paper/2309.00121/) (2023)
- [Primus: Enforcing Attention Usage for 3D Medical Image Segmentation](https://mlsanity.com/paper/2503.01835/) (2025)
- [3D Vision with Transformers: A Survey](https://mlsanity.com/paper/2208.04309/) (2022)
- [BATFormer: Towards Boundary-Aware Lightweight Transformer for Efficient Medical Image Segmentation](https://mlsanity.com/paper/2206.14409/) (2022)
- [UniBrain: Universal Brain MRI Diagnosis with Hierarchical Knowledge-enhanced Pre-training](https://mlsanity.com/paper/2309.06828/) (2023)
- [Laplacian-Former: Overcoming the Limitations of Vision Transformers in Local Texture Detection](https://mlsanity.com/paper/2309.00108/) (2023)
- [A Comprehensive Review of U-Net and Its Variants: Advances and Applications in Medical Image Segmentation](https://mlsanity.com/paper/2502.06895/) (2025)
- [The Lighter The Better: Rethinking Transformers in Medical Image Segmentation Through Adaptive Pruning](https://mlsanity.com/paper/2206.14413/) (2022)
- [Recent Progress in Transformer-based Medical Image Analysis](https://mlsanity.com/paper/2208.06643/) (2022)
- [STPNet: Scale-aware Text Prompt Network for Medical Image Segmentation](https://mlsanity.com/paper/2504.01561/) (2025)
- [Understanding the Tricks of Deep Learning in Medical Image Segmentation: Challenges and Future Directions](https://mlsanity.com/paper/2209.10307/) (2022)
- [HAFormer: Unleashing the Power of Hierarchy-Aware Features for Lightweight Semantic Segmentation](https://mlsanity.com/paper/2407.07441/) (2024)
- [Token Sparsification for Faster Medical Image Segmentation](https://mlsanity.com/paper/2303.06522/) (2023)
- [Multiscale Progressive Text Prompt Network for Medical Image Segmentation](https://mlsanity.com/paper/2307.00174/) (2023)
- [SegNetr: Rethinking the local-global interactions and skip connections in U-shaped networks](https://mlsanity.com/paper/2307.02953/) (2023)
- [AI-Enhanced Virtual Reality in Medicine: A Comprehensive Survey](https://mlsanity.com/paper/2402.03093/) (2024)
- [UNesT: Local Spatial Representation Learning with Hierarchical Transformer for Efficient Medical Segmentation](https://mlsanity.com/paper/2209.14378/) (2022)
- [Self-supervised cost of transport estimation for multimodal path planning](https://mlsanity.com/paper/2412.06101/) (2024)
- [Trans2-CBCT: A Dual-Transformer Framework for Sparse-View CBCT Reconstruction](https://mlsanity.com/paper/2506.17425/) (2025)
- [Model Agnostic Preference Optimization for Medical Image Segmentation](https://mlsanity.com/paper/2512.15009/) (2025)
- [OMSN and FAROS: OCTA Microstructure Segmentation Network and Fully Annotated Retinal OCTA Segmentation Dataset](https://mlsanity.com/paper/2212.13059/) (2022)
- [MAF-Net: Multiple attention-guided fusion network for fundus vascular image segmentation](https://mlsanity.com/paper/2305.03617/) (2023)

## Abstract (arXiv)

Computer-aided medical image segmentation has been applied widely in diagnosis and treatment to obtain clinically useful information of shapes and volumes of target organs and tissues. In the past several years, convolutional neural network (CNN) based methods (e.g., U-Net) have dominated this area, but still suffered from inadequate long-range information capturing. Hence, recent work presented computer vision Transformer variants for medical image segmentation tasks and obtained promising performances. Such Transformers model long-range dependency by computing pair-wise patch relations. However, they incur prohibitive computational costs, especially on 3D medical images (e.g., CT and MRI). In this paper, we propose a new method called Dilated Transformer, which conducts self-attention for pair-wise patch relations captured alternately in local and global scopes. Inspired by dilated convolution kernels, we conduct the global self-attention in a dilated manner, enlarging receptive fields without increasing the patches involved and thus reducing computational costs. Based on this design of Dilated Transformer, we construct a U-shaped encoder-decoder hierarchical architecture called D-Former for 3D medical image segmentation. Experiments on the Synapse and ACDC datasets show that our D-Former model, trained from scratch, outperforms various competitive CNN-based or Transformer-based segmentation models at a low computational cost without time-consuming per-training process.