# SegStitch: Multidimensional Transformer for Robust and Efficient Medical Imaging Segmentation

arXiv: https://arxiv.org/abs/2408.00496 · HTML: https://mlsanity.com/paper/2408.00496/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 61 references that are in the map.
- First posted on arXiv 1 Aug 2024.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation](https://mlsanity.com/paper/2004.08790/) (2020)
- [Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images](https://mlsanity.com/paper/1808.08114/) (2018)
- [TransBTS: Multimodal Brain Tumor Segmentation Using Transformer](https://mlsanity.com/paper/2103.04430/) (2021)
- [TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation](https://mlsanity.com/paper/2102.08005/) (2021)
- [MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation](https://mlsanity.com/paper/1902.04049/) (2019)
- [Context Prior for Scene Segmentation](https://mlsanity.com/paper/2004.01547/) (2020)
- [UNETR++: Delving into Efficient and Accurate 3D Medical Image Segmentation](https://mlsanity.com/paper/2212.04497/) (2022)
- [On the Adversarial Robustness of Vision Transformers](https://mlsanity.com/paper/2103.15670/) (2021)
- [MISSFormer: An Effective Medical Image Segmentation Transformer](https://mlsanity.com/paper/2109.07162/) (2021)
- [Stable Neural ODE with Lyapunov-Stable Equilibrium Points for Defending Against Adversarial Attacks](https://mlsanity.com/paper/2110.12976/) (2021)
- [LeViT-UNet: Make Faster Encoders with Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2107.08623/) (2021)
- [On Robustness of Neural Ordinary Differential Equations](https://mlsanity.com/paper/1910.05513/) (2019)
- [Abdominal multi-organ segmentation with organ-attention networks and statistical fusion](https://mlsanity.com/paper/1804.08414/) (2018)
- [TransDeepLab: Convolution-Free Transformer-based DeepLab v3+ for Medical Image Segmentation](https://mlsanity.com/paper/2208.00713/) (2022)
- [TransClaw U-Net: Claw U-Net with Transformers for Medical Image Segmentation](https://mlsanity.com/paper/2107.05188/) (2021)
- [TransNorm: Transformer Provides a Strong Spatial Normalization Mechanism for a Deep Segmentation Model](https://mlsanity.com/paper/2207.13415/) (2022)
- [BHSD: A 3D Multi-Class Brain Hemorrhage Segmentation Dataset](https://mlsanity.com/paper/2308.11298/) (2023)
- [Efficient Learning With Sine-Activated Low-rank Matrices](https://mlsanity.com/paper/2403.19243/) (2024)
- [UNetFormer: A Unified Vision Transformer Model and Pre-Training Framework for 3D Medical Image Segmentation](https://mlsanity.com/paper/2204.00631/) (2022)

## Built on by (most cited first, 13 of 13)

- [MedConv: Convolutions Beat Transformers on Long-Tailed Bone Density Prediction](https://mlsanity.com/paper/2502.00631/) (2025)
- [ESA: Annotation-Efficient Active Learning for Semantic Segmentation](https://mlsanity.com/paper/2408.13491/) (2024)
- [MSDet: Receptive Field Enhanced Multiscale Detection for Tiny Pulmonary Nodule](https://mlsanity.com/paper/2409.14028/) (2024)
- [MedDet: Generative Adversarial Distillation for Efficient Cervical Disc Herniation Detection](https://mlsanity.com/paper/2409.00204/) (2024)
- [GAMED-Snake: Gradient-aware Adaptive Momentum Evolution Deep Snake Model for Multi-organ Segmentation](https://mlsanity.com/paper/2501.12844/) (2025)
- [PathoHR: Breast Cancer Survival Prediction on High-Resolution Pathological Images](https://mlsanity.com/paper/2503.17970/) (2025)
- [ProjectedEx: Enhancing Generation in Explainable AI for Prostate Cancer](https://mlsanity.com/paper/2501.01392/) (2025)
- [Medical Artificial Intelligence for Early Detection of Lung Cancer: A Survey](https://mlsanity.com/paper/2410.14769/) (2024)
- [DOEI: Dual Optimization of Embedding Information for Attention-Enhanced Class Activation Maps](https://mlsanity.com/paper/2502.15885/) (2025)
- [SSS: Semi-Supervised SAM-2 with Efficient Prompting for Medical Imaging Segmentation](https://mlsanity.com/paper/2506.08949/) (2025)
- [A Novel Convolutional-Free Method for 3D Medical Imaging Segmentation](https://mlsanity.com/paper/2502.05396/) (2025)
- [3D Medical Imaging Segmentation on Non-Contrast CT](https://mlsanity.com/paper/2503.08361/) (2025)
- [SegTTA: Training-Free Test-Time Augmentation for Zero-Shot Medical Imaging Segmentation](https://mlsanity.com/paper/2604.17451/) (2026)

## Abstract (arXiv)

Medical imaging segmentation plays a significant role in the automatic recognition and analysis of lesions. State-of-the-art methods, particularly those utilizing transformers, have been prominently adopted in 3D semantic segmentation due to their superior performance in scalability and generalizability. However, plain vision transformers encounter challenges due to their neglect of local features and their high computational complexity. To address these challenges, we introduce three key contributions: Firstly, we proposed SegStitch, an innovative architecture that integrates transformers with denoising ODE blocks. Instead of taking whole 3D volumes as inputs, we adapt axial patches and customize patch-wise queries to ensure semantic consistency. Additionally, we conducted extensive experiments on the BTCV and ACDC datasets, achieving improvements up to 11.48% and 6.71% respectively in mDSC, compared to state-of-the-art methods. Lastly, our proposed method demonstrates outstanding efficiency, reducing the number of parameters by 36.7% and the number of FLOPS by 10.7% compared to UNETR. This advancement holds promising potential for adapting our method to real-world clinical practice. The code will be available at https://github.com/goblin327/SegStitch