# Large-Kernel Attention for 3D Medical Image Segmentation

arXiv: https://arxiv.org/abs/2207.11225 · HTML: https://mlsanity.com/paper/2207.11225/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 63 references that are in the map.
- First posted on arXiv 19 Jul 2022.

## Builds on (most cited first, 30 of 36)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge](https://mlsanity.com/paper/1811.02629/) (2018)
- [The Liver Tumor Segmentation Benchmark (LiTS)](https://mlsanity.com/paper/1901.04056/) (2019)
- [Residual Attention Network for Image Classification](https://mlsanity.com/paper/1704.06904/) (2017)
- [3D MRI brain tumor segmentation using autoencoder regularization](https://mlsanity.com/paper/1810.11654/) (2018)
- [Medical Transformer: Gated Axial-Attention for Medical Image Segmentation](https://mlsanity.com/paper/2102.10662/) (2021)
- [Attention Mechanisms in Computer Vision: A Survey](https://mlsanity.com/paper/2111.07624/) (2021)
- [Visual Attention Network](https://mlsanity.com/paper/2202.09741/) (2022)
- [Large Kernel Matters -- Improve Semantic Segmentation by Global Convolutional Network](https://mlsanity.com/paper/1703.02719/) (2017)
- [BAM: Bottleneck Attention Module](https://mlsanity.com/paper/1807.06514/) (2018)
- [Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks](https://mlsanity.com/paper/1810.12348/) (2018)
- [nnU-Net for Brain Tumor Segmentation](https://mlsanity.com/paper/2011.00848/) (2020)
- [Multi-scale self-guided attention for medical image segmentation](https://mlsanity.com/paper/1906.02849/) (2019)
- [Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyond](https://mlsanity.com/paper/2102.01998/) (2021)
- [Optimized U-Net for Brain Tumor Segmentation](https://mlsanity.com/paper/2110.03352/) (2021)
- [Extending nn-UNet for brain tumor segmentation](https://mlsanity.com/paper/2112.04653/) (2021)
- [An application of cascaded 3D fully convolutional networks for medical image segmentation](https://mlsanity.com/paper/1803.05431/) (2018)
- [TBraTS: Trusted Brain Tumor Segmentation](https://mlsanity.com/paper/2206.09309/) (2022)
- [Cascaded V-Net using ROI masks for brain tumor segmentation](https://mlsanity.com/paper/1812.11588/) (2018)
- [Data Harmonisation for Information Fusion in Digital Healthcare: A State-of-the-Art Systematic Review, Meta-Analysis and Future Research Directions](https://mlsanity.com/paper/2201.06505/) (2022)
- [Modality-Pairing Learning for Brain Tumor Segmentation](https://mlsanity.com/paper/2010.09277/) (2020)
- [MRI Tumor Segmentation with Densely Connected 3D CNN](https://mlsanity.com/paper/1802.02427/) (2018)
- [Spatial Context-Aware Self-Attention Model For Multi-Organ Segmentation](https://mlsanity.com/paper/2012.09279/) (2020)
- [Human Treelike Tubular Structure Segmentation: A Comprehensive Review and Future Perspectives](https://mlsanity.com/paper/2207.11203/) (2022)
- [H2NF-Net for Brain Tumor Segmentation using Multimodal MR Imaging: 2nd Place Solution to BraTS Challenge 2020 Segmentation Task](https://mlsanity.com/paper/2012.15318/) (2020)
- [3D AGSE-VNet: An Automatic Brain Tumor MRI Data Segmentation Framework](https://mlsanity.com/paper/2107.12046/) (2021)

## Built on by (most cited first, 7 of 7)

- [MedNeXt: Transformer-driven Scaling of ConvNets for Medical Image Segmentation](https://mlsanity.com/paper/2303.09975/) (2023)
- [Region-Based Evidential Deep Learning to Quantify Uncertainty and Improve Robustness of Brain Tumor Segmentation](https://mlsanity.com/paper/2208.06038/) (2022)
- [Scaling Up 3D Kernels with Bayesian Frequency Re-parameterization for Medical Image Segmentation](https://mlsanity.com/paper/2303.05785/) (2023)
- [MIST: Medical Image Segmentation Transformer with Convolutional Attention Mixing (CAM) Decoder](https://mlsanity.com/paper/2310.19898/) (2023)
- [VSR-Net: Vessel-like Structure Rehabilitation Network with Graph Clustering](https://mlsanity.com/paper/2312.13116/) (2023)
- [SchoenbAt: Rethinking Attention with Polynomial basis](https://mlsanity.com/paper/2505.12252/) (2025)
- [Vision Transformers increase efficiency of 3D cardiac CT multi-label segmentation](https://mlsanity.com/paper/2310.09099/) (2023)

## Abstract (arXiv)

Automatic segmentation of multiple organs and tumors from 3D medical images such as magnetic resonance imaging (MRI) and computed tomography (CT) scans using deep learning methods can aid in diagnosing and treating cancer. However, organs often overlap and are complexly connected, characterized by extensive anatomical variation and low contrast. In addition, the diversity of tumor shape, location, and appearance, coupled with the dominance of background voxels, makes accurate 3D medical image segmentation difficult. In this paper, a novel large-kernel (LK) attention module is proposed to address these problems to achieve accurate multi-organ segmentation and tumor segmentation. The advantages of convolution and self-attention are combined in the proposed LK attention module, including local contextual information, long-range dependence, and channel adaptation. The module also decomposes the LK convolution to optimize the computational cost and can be easily incorporated into FCNs such as U-Net. Comprehensive ablation experiments demonstrated the feasibility of convolutional decomposition and explored the most efficient and effective network design. Among them, the best Mid-type LK attention-based U-Net network was evaluated on CT-ORG and BraTS 2020 datasets, achieving state-of-the-art segmentation performance. The performance improvement due to the proposed LK attention module was also statistically validated.