# A Robust Volumetric Transformer for Accurate 3D Tumor Segmentation

arXiv: https://arxiv.org/abs/2111.13300 · HTML: https://mlsanity.com/paper/2111.13300/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 30 references that are in the map.
- First posted on arXiv 26 Nov 2021.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [The Medical Segmentation Decathlon](https://mlsanity.com/paper/2106.05735/) (2021)
- [Intriguing Properties of Vision Transformers](https://mlsanity.com/paper/2105.10497/) (2021)
- [TransBTS: Multimodal Brain Tumor Segmentation Using Transformer](https://mlsanity.com/paper/2103.04430/) (2021)
- [CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.03024/) (2021)
- [Local Relation Networks for Image Recognition](https://mlsanity.com/paper/1904.11491/) (2019)
- [nnU-Net for Brain Tumor Segmentation](https://mlsanity.com/paper/2011.00848/) (2020)

## Built on by (most cited first, 30 of 41)

- [MedNeXt: Transformer-driven Scaling of ConvNets for Medical Image Segmentation](https://mlsanity.com/paper/2303.09975/) (2023)
- [SwinMM: Masked Multi-view with Swin Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2307.12591/) (2023)
- [Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives](https://mlsanity.com/paper/2206.01136/) (2022)
- [CKD-TransBTS: Clinical Knowledge-Driven Hybrid Transformer with Modality-Correlated Cross-Attention for Brain Tumor Segmentation](https://mlsanity.com/paper/2207.07370/) (2022)
- [Primus: Enforcing Attention Usage for 3D Medical Image Segmentation](https://mlsanity.com/paper/2503.01835/) (2025)
- [A Recent Survey of Vision Transformers for Medical Image Segmentation](https://mlsanity.com/paper/2312.00634/) (2023)
- [3D TransUNet: Advancing Medical Image Segmentation through Vision Transformers](https://mlsanity.com/paper/2310.07781/) (2023)
- [A survey on attention mechanisms for medical applications: are we moving towards better algorithms?](https://mlsanity.com/paper/2204.12406/) (2022)
- [EM-Net: Efficient Channel and Frequency Learning with Mamba for 3D Medical Image Segmentation](https://mlsanity.com/paper/2409.17675/) (2024)
- [Recent Progress in Transformer-based Medical Image Analysis](https://mlsanity.com/paper/2208.06643/) (2022)
- [D-Net: Dynamic Large Kernel with Dynamic Feature Fusion for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2403.10674/) (2024)
- [Rethinking Brain Tumor Segmentation from the Frequency Domain Perspective](https://mlsanity.com/paper/2506.10142/) (2025)
- [CSAM: A 2.5D Cross-Slice Attention Module for Anisotropic Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2311.04942/) (2023)
- [Multi-branch Cascaded Swin Transformers with Attention to k-space Sampling Pattern for Accelerated MRI Reconstruction](https://mlsanity.com/paper/2207.08412/) (2022)
- [Improving Robustness and Reliability in Medical Image Classification with Latent-Guided Diffusion and Nested-Ensembles](https://mlsanity.com/paper/2310.15952/) (2023)
- [PIP: Positional-encoding Image Prior](https://mlsanity.com/paper/2211.14298/) (2022)
- [Boosting Convolution with Efficient MLP-Permutation for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2303.13111/) (2023)
- [RadarFormer: Lightweight and Accurate Real-Time Radar Object Detection Model](https://mlsanity.com/paper/2304.08447/) (2023)
- [Med-Tuning: A New Parameter-Efficient Tuning Framework for Medical Volumetric Segmentation](https://mlsanity.com/paper/2304.10880/) (2023)
- [Concentrate on Weakness: Mining Hard Prototypes for Few-Shot Medical Image Segmentation](https://mlsanity.com/paper/2505.21897/) (2025)
- [Transformer-based end-to-end classification of variable-length volumetric data](https://mlsanity.com/paper/2307.06666/) (2023)
- [Deep evidential fusion with uncertainty quantification and contextual discounting for multimodal medical image segmentation](https://mlsanity.com/paper/2309.05919/) (2023)
- [Architecture Analysis and Benchmarking of 3D U-shaped Deep Learning Models for Thoracic Anatomical Segmentation](https://mlsanity.com/paper/2402.03230/) (2024)
- [DocParseNet: Advanced Semantic Segmentation and OCR Embeddings for Efficient Scanned Document Annotation](https://mlsanity.com/paper/2406.17591/) (2024)
- [Cross-Slice Attention and Evidential Critical Loss for Uncertainty-Aware Prostate Cancer Detection](https://mlsanity.com/paper/2407.01146/) (2024)
- [Spectral U-Net: Enhancing Medical Image Segmentation via Spectral Decomposition](https://mlsanity.com/paper/2409.09216/) (2024)
- [GBT-SAM: A Parameter-Efficient Depth-Aware Model for Generalizable Brain tumour Segmentation on mp-MRI](https://mlsanity.com/paper/2503.04325/) (2025)
- [UKAN-EP: Enhancing U-KAN with Efficient Attention and Pyramid Aggregation for 3D Multi-Modal MRI Brain Tumor Segmentation](https://mlsanity.com/paper/2408.00273/) (2024)
- [QMaxViT-Unet+: A Query-Based MaxViT-Unet with Edge Enhancement for Scribble-Supervised Segmentation of Medical Images](https://mlsanity.com/paper/2502.10294/) (2025)
- [Trans2-CBCT: A Dual-Transformer Framework for Sparse-View CBCT Reconstruction](https://mlsanity.com/paper/2506.17425/) (2025)

## Abstract (arXiv)

We propose a Transformer architecture for volumetric segmentation, a challenging task that requires keeping a complex balance in encoding local and global spatial cues, and preserving information along all axes of the volume. Encoder of the proposed design benefits from self-attention mechanism to simultaneously encode local and global cues, while the decoder employs a parallel self and cross attention formulation to capture fine details for boundary refinement. Empirically, we show that the proposed design choices result in a computationally efficient model, with competitive and promising results on the Medical Segmentation Decathlon (MSD) brain tumor segmentation (BraTS) Task. We further show that the representations learned by our model are robust against data corruptions. hrefhttps://github.com/himashi92/VT-UNetOur code implementation is publicly available.