# Harnessing The Power of Attention For Patch-Based Biomedical Image Classification

arXiv: https://arxiv.org/abs/2404.00949 · HTML: https://mlsanity.com/paper/2404.00949/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 44 references that are in the map.
- First posted on arXiv 1 Apr 2024.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Linformer: Self-Attention with Linear Complexity](https://mlsanity.com/paper/2006.04768/) (2020)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [DeiT III: Revenge of the ViT](https://mlsanity.com/paper/2204.07118/) (2022)
- [Convolutional Two-Stream Network Fusion for Video Action Recognition](https://mlsanity.com/paper/1604.06573/) (2016)
- [Attention Augmented Convolutional Networks](https://mlsanity.com/paper/1904.09925/) (2019)
- [Relation Networks for Object Detection](https://mlsanity.com/paper/1711.11575/) (2017)
- [Towards Generalist Biomedical AI](https://mlsanity.com/paper/2307.14334/) (2023)
- [Local Relation Networks for Image Recognition](https://mlsanity.com/paper/1904.11491/) (2019)
- [Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution](https://mlsanity.com/paper/1904.05049/) (2019)
- [I-ViT: Integer-only Quantization for Efficient Vision Transformer Inference](https://mlsanity.com/paper/2207.01405/) (2022)
- [SOLQ: Segmenting Objects by Learning Queries](https://mlsanity.com/paper/2106.02351/) (2021)
- [ViTO: Vision Transformer-Operator](https://mlsanity.com/paper/2303.08891/) (2023)
- [Big-Little Net: An Efficient Multi-Scale Feature Representation for Visual and Speech Recognition](https://mlsanity.com/paper/1807.03848/) (2018)
- [FastSurferVINN: Building Resolution-Independence into Deep Learning Segmentation Methods -- A Solution for HighRes Brain MRI](https://mlsanity.com/paper/2112.09654/) (2021)

## Abstract (arXiv)

Biomedical image analysis is of paramount importance for the advancement of healthcare and medical research. Although conventional convolutional neural networks (CNNs) are frequently employed in this domain, facing limitations in capturing intricate spatial and temporal relationships at the pixel level due to their reliance on fixed-sized windows and immutable filter weights post-training. These constraints impede their ability to adapt to input fluctuations and comprehend extensive long-range contextual information. To overcome these challenges, a novel architecture based on self-attention mechanisms as an alternative to conventional CNNs.The proposed model utilizes attention-based mechanisms to surpass the limitations of CNNs. The key component of our strategy is the combination of non-overlapping (vanilla patching) and novel overlapped Shifted Patching Techniques (S.P.T.s), which enhances the model's capacity to capture local context and improves generalization. Additionally, we introduce the Lancoz5 interpolation technique, which adapts variable image sizes to higher resolutions, facilitating better analysis of high-resolution biomedical images. Our methods address critical challenges faced by attention-based vision models, including inductive bias, weight sharing, receptive field limitations, and efficient data handling. Experimental evidence shows the effectiveness of proposed model in generalizing to various biomedical imaging tasks. The attention-based model, combined with advanced data augmentation methodologies, exhibits robust modeling capabilities and superior performance compared to existing approaches. The integration of S.P.T.s significantly enhances the model's ability to capture local context, while the Lancoz5 interpolation technique ensures efficient handling of high-resolution images.