Memory transformers for full context and high-resolution 3D Medical Segmentation
Not on the map. Why?
Abstract (arXiv)
Transformer models achieve state-of-the-art results for image segmentation. However, achieving long-range attention, necessary to capture global context, with high-resolution 3D images is a fundamental challenge. This paper introduces the Full resolutIoN mEmory (FINE) transformer to overcome this issue. The core idea behind FINE is to learn memory tokens to indirectly model full range interactions while scaling well in both memory and computational costs. FINE introduces memory tokens at two levels: the first one allows full interaction between voxels within local image regions (patches), the second one allows full interactions between all regions of the 3D volume. Combined, they allow full attention over high resolution images, e.g. 512 x 512 x 256 voxels and above. Experiments on the BCV image segmentation dataset shows better performances than state-of-the-art CNN and transformer baselines, highlighting the superiority of our full attention mechanism compared to recent transformer baselines, e.g. CoTr, and nnFormer.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 28 of its 34 references that are in the map. First posted on arXiv 11 Oct 2022.
Builds on
28 of 34 references.
References · 28
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
- 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
- 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
- 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
- 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,552
- 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
- 2018Attention U-Net: Learning Where to Look for the Pancreascited by 1,420
- 2021Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentationcited by 1,218
16 more
- 2021UNETR: Transformers for 3D Medical Image Segmentationcited by 954
- 2021Multiscale Vision Transformerscited by 896
- 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 749
- 2019Compressive Transformers for Long-Range Sequence Modellingcited by 699
- 2017Understanding the Effective Receptive Field in Deep Convolutional Neural Networkscited by 549
- 2018Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challengecited by 508
- 2021Random Feature Attentioncited by 350
- 2021Medical Transformer: Gated Axial-Attention for Medical Image Segmentationcited by 319
- 2016Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentationcited by 263
- 2021CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentationcited by 227
- 2021Multi-Scale Vision Longformer: A New Vision Transformer for High-Resolution Image Encodingcited by 207
- 2019Blockwise Self-Attention for Long Document Understandingcited by 158
- 2021Video Instance Segmentation using Inter-Frame Communication Transformerscited by 105
- 2021Convolution-Free Medical Image Segmentation using Transformerscited by 49
- 2021Boundary-aware Transformers for Skin Lesion Segmentationcited by 39
- 2021Ultrasound Video Transformers for Cardiac Ejection Fraction Estimationcited by 34
Built on by
Citing papers per year: 2023: 2; 2024: 0; 2025: 0; 2026: 1.
3 papers.