mlsanityData to 14 Oct 2026

Large-kernel Attention for Efficient and Robust Brain Lesion Segmentation

arXiv 2308.07251 ↗First posted 14 Aug 2023eess.IV cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Vision transformers are effective deep learning models for vision tasks, including medical image segmentation. However, they lack efficiency and translational invariance, unlike convolutional neural networks (CNNs). To model long-range interactions in 3D brain lesion segmentation, we propose an all-convolutional transformer block variant of the U-Net architecture. We demonstrate that our model provides the greatest compromise in three factors: performance competitive with the state-of-the-art; parameter efficiency of a CNN; and the favourable inductive biases of a transformer. Our public implementation is available at https://github.com/liamchalcroft/MDUNet .

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 27 of its 44 references that are in the map. First posted on arXiv 14 Aug 2023.

Builds on

27 of 44 references.

References · 27

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  4. 2016Layer Normalizationcited by 5,669
  5. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  6. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  7. 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,558
  8. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
  9. 2022Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Imagescited by 830
  10. 2021Going deeper with Image Transformerscited by 764
  11. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 751
  12. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 675
15 more
  1. 2021MetaFormer Is Actually What You Need for Visioncited by 524
  2. 2021The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classificationcited by 518
  3. 2019Why gradient clipping accelerates training: A theoretical justification for adaptivitycited by 341
  4. 2022SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentationcited by 252
  5. 2022Visual Attention Networkcited by 210
  6. 2022Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNscited by 155
  7. 2022More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsitycited by 142
  8. 2021Are Convolutional Neural Networks or Transformers more like human vision?cited by 109
  9. 2022ISLES 2022: A multi-center magnetic resonance imaging stroke lesion segmentation datasetcited by 106
  10. 20223D UX-Net: A Large Kernel Volumetric ConvNet Modernizing Hierarchical Transformer for Medical Image Segmentationcited by 94
  11. 2021Optimized U-Net for Brain Tumor Segmentationcited by 40
  12. 2022U-Net vs Transformer: Is U-Net Outdated in Medical Image Registration?cited by 40
  13. 2020On the Texture Bias for Few-Shot CNN Segmentationcited by 38
  14. 2022Factorizer: A Scalable Interpretable Approach to Context Modeling for Medical Image Segmentationcited by 12
  15. 2019Texture Bias Of CNNs Limits Few-Shot Classification Performancecited by 3

Built on by

Cited by year

Citing papers per year: 2024: 4; 2025: 1.

5 papers.

  1. 2024Imbalance-Aware Culvert-Sewer Defect Segmentation Using an Enhanced Feature Pyramid Networkcited by 7
  2. 2024Transformers-based architectures for stroke segmentation: A reviewcited by 5
  3. 2024Deep models for stroke segmentation: do complex architectures always perform better?cited by 3
  4. 2025Unified 3D MRI Representations via Sequence-Invariant Contrastive Learningcited by 1
  5. 2024Neuro-TransUNet: Segmentation of stroke lesion in MRI using transformerscited by 0