# Studying the Effects of Self-Attention for Medical Image Analysis

arXiv: https://arxiv.org/abs/2109.01486 · HTML: https://mlsanity.com/paper/2109.01486/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 35 references that are in the map.
- First posted on arXiv 2 Sep 2021.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [CCNet: Criss-Cross Attention for Semantic Segmentation](https://mlsanity.com/paper/1811.11721/) (2018)
- [CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning](https://mlsanity.com/paper/1711.05225/) (2017)
- [Deep Learning for Medical Image Analysis](https://mlsanity.com/paper/1708.08987/) (2017)
- [Residual Attention Network for Image Classification](https://mlsanity.com/paper/1704.06904/) (2017)
- [GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond](https://mlsanity.com/paper/1904.11492/) (2019)
- [COVID-CT-Dataset: A CT Scan Dataset about COVID-19](https://mlsanity.com/paper/2003.13865/) (2020)
- [SRM : A Style-based Recalibration Module for Convolutional Neural Networks](https://mlsanity.com/paper/1903.10829/) (2019)
- [Residual and Plain Convolutional Neural Networks for 3D Brain MRI Classification](https://mlsanity.com/paper/1701.06643/) (2017)
- [RADNET: Radiologist Level Accuracy using Deep Learning for HEMORRHAGE detection in CT Scans](https://mlsanity.com/paper/1710.04934/) (2017)

## Built on by (most cited first, 6 of 6)

- [Pyramid Pixel Context Adaption Network for Medical Image Classification with Supervised Contrastive Learning](https://mlsanity.com/paper/2303.01917/) (2023)
- [SAF-Net: Self-Attention Fusion Network for Myocardial Infarction Detection using Multi-View Echocardiography](https://mlsanity.com/paper/2309.15520/) (2023)
- [CDNet: Contrastive Disentangled Network for Fine-Grained Image Categorization of Ocular B-Scan Ultrasound](https://mlsanity.com/paper/2206.08524/) (2022)
- [RevealED: Uncovering Pro-Eating Disorder Content on Twitter Using Deep Learning](https://mlsanity.com/paper/2212.13949/) (2022)
- [A Novel Momentum-Based Deep Learning Techniques for Medical Image Classification and Segmentation](https://mlsanity.com/paper/2408.05692/) (2024)
- [Studying the Effects of Self-Attention on SAR Automatic Target Recognition](https://mlsanity.com/paper/2409.00473/) (2024)

## Abstract (arXiv)

When the trained physician interprets medical images, they understand the clinical importance of visual features. By applying cognitive attention, they apply greater focus onto clinically relevant regions while disregarding unnecessary features. The use of computer vision to automate the classification of medical images is widely studied. However, the standard convolutional neural network (CNN) does not necessarily employ subconscious feature relevancy evaluation techniques similar to the trained medical specialist and evaluates features more generally. Self-attention mechanisms enable CNNs to focus more on semantically important regions or aggregated relevant context with long-range dependencies. By using attention, medical image analysis systems can potentially become more robust by focusing on more important clinical feature regions. In this paper, we provide a comprehensive comparison of various state-of-the-art self-attention mechanisms across multiple medical image analysis tasks. Through both quantitative and qualitative evaluations along with a clinical user-centric survey study, we aim to provide a deeper understanding of the effects of self-attention in medical computer vision tasks.