# Spatial Context-Aware Self-Attention Model For Multi-Organ Segmentation

arXiv: https://arxiv.org/abs/2012.09279 · HTML: https://mlsanity.com/paper/2012.09279/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Dec 2020.

## Built on by (most cited first, 16 of 16)

- [CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection](https://mlsanity.com/paper/2301.00785/) (2023)
- [SimCVD: Simple Contrastive Voxel-Wise Representation Distillation for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2108.06227/) (2021)
- [Recurrent Mask Refinement for Few-Shot Medical Image Segmentation](https://mlsanity.com/paper/2108.00622/) (2021)
- [AFTer-UNet: Axial Fusion Transformer UNet for Medical Image Segmentation](https://mlsanity.com/paper/2110.10403/) (2021)
- [Topology-Preserving Shape Reconstruction and Registration via Neural Diffeomorphic Flow](https://mlsanity.com/paper/2203.08652/) (2022)
- [TransFusion: Cross-view Fusion with Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2110.09554/) (2021)
- [Diffeomorphic Image Registration with Neural Velocity Field](https://mlsanity.com/paper/2202.12498/) (2022)
- [MEGAN: Memory Enhanced Graph Attention Network for Space-Time Video Super-Resolution](https://mlsanity.com/paper/2110.15327/) (2021)
- [Recent Advances in Medical Imaging Segmentation: A Survey](https://mlsanity.com/paper/2505.09274/) (2025)
- [A Novel Hybrid Convolutional Neural Network for Accurate Organ Segmentation in 3D Head and Neck CT Images](https://mlsanity.com/paper/2109.12634/) (2021)
- [Large-Kernel Attention for 3D Medical Image Segmentation](https://mlsanity.com/paper/2207.11225/) (2022)
- [Tackling the Class Imbalance Problem of Deep Learning Based Head and Neck Organ Segmentation](https://mlsanity.com/paper/2201.01636/) (2022)
- [Towards more precise automatic analysis: a comprehensive survey of deep learning-based multi-organ segmentation](https://mlsanity.com/paper/2303.00232/) (2023)
- [Multi-organ segmentation: a progressive exploration of learning paradigms under scarce annotation](https://mlsanity.com/paper/2302.03296/) (2023)
- [Hierarchical Attention for Sparse Volumetric Anomaly Detection in Subclinical Keratoconus](https://mlsanity.com/paper/2512.03346/) (2025)
- [Large Batch and Patch Size Training for Medical Image Segmentation](https://mlsanity.com/paper/2210.13364/) (2022)

## Abstract (arXiv)

Multi-organ segmentation is one of most successful applications of deep learning in medical image analysis. Deep convolutional neural nets (CNNs) have shown great promise in achieving clinically applicable image segmentation performance on CT or MRI images. State-of-the-art CNN segmentation models apply either 2D or 3D convolutions on input images, with pros and cons associated with each method: 2D convolution is fast, less memory-intensive but inadequate for extracting 3D contextual information from volumetric images, while the opposite is true for 3D convolution. To fit a 3D CNN model on CT or MRI images on commodity GPUs, one usually has to either downsample input images or use cropped local regions as inputs, which limits the utility of 3D models for multi-organ segmentation. In this work, we propose a new framework for combining 3D and 2D models, in which the segmentation is realized through high-resolution 2D convolutions, but guided by spatial contextual information extracted from a low-resolution 3D model. We implement a self-attention mechanism to control which 3D features should be used to guide 2D segmentation. Our model is light on memory usage but fully equipped to take 3D contextual information into account. Experiments on multiple organ segmentation datasets demonstrate that by taking advantage of both 2D and 3D models, our method is consistently outperforms existing 2D and 3D models in organ segmentation accuracy, while being able to directly take raw whole-volume image data as inputs.