# SA2-Net: Scale-aware Attention Network for Microscopic Image Segmentation

arXiv: https://arxiv.org/abs/2309.16661 · HTML: https://mlsanity.com/paper/2309.16661/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 39 references that are in the map.
- First posted on arXiv 28 Sep 2023.

## Builds on (most cited first, 30 of 32)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1902.03368/) (2019)
- [Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs](https://mlsanity.com/paper/1412.7062/) (2014)
- [Road Extraction by Deep Residual U-Net](https://mlsanity.com/paper/1711.10684/) (2017)
- [Medical Transformer: Gated Axial-Attention for Medical Image Segmentation](https://mlsanity.com/paper/2102.10662/) (2021)
- [TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation](https://mlsanity.com/paper/2102.08005/) (2021)
- [Transformers in Medical Imaging: A Survey](https://mlsanity.com/paper/2201.09873/) (2022)
- [CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.03024/) (2021)
- [Gland Segmentation in Colon Histology Images: The GlaS Challenge Contest](https://mlsanity.com/paper/1603.00275/) (2016)
- [UCTransNet: Rethinking the Skip Connections in U-Net from a Channel-wise Perspective with Transformer](https://mlsanity.com/paper/2109.04335/) (2021)
- [MultiResUNet : Rethinking the U-Net Architecture for Multimodal Biomedical Image Segmentation](https://mlsanity.com/paper/1902.04049/) (2019)
- [Medical Image Segmentation Review: The success of U-Net](https://mlsanity.com/paper/2211.14830/) (2022)
- [DS-TransUNet:Dual Swin Transformer U-Net for Medical Image Segmentation](https://mlsanity.com/paper/2106.06716/) (2021)
- [MISSFormer: An Effective Medical Image Segmentation Transformer](https://mlsanity.com/paper/2109.07162/) (2021)
- [Multi-scale self-guided attention for medical image segmentation](https://mlsanity.com/paper/1906.02849/) (2019)
- [A Data-scalable Transformer for Medical Image Segmentation: Architecture, Model Efficiency, and Benchmark](https://mlsanity.com/paper/2203.00131/) (2022)
- [DCSAU-Net: A Deeper and More Compact Split-Attention U-Net for Medical Image Segmentation](https://mlsanity.com/paper/2202.00972/) (2022)
- [Multi-level Context Gating of Embedded Collective Knowledge for Medical Image Segmentation](https://mlsanity.com/paper/2003.05056/) (2020)
- [Pyramid Medical Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2104.14702/) (2021)
- [A Multi-Branch Hybrid Transformer Networkfor Corneal Endothelial Cell Segmentation](https://mlsanity.com/paper/2106.07557/) (2021)

## Built on by (most cited first, 6 of 6)

- [Vision-Language Synthetic Data Enhances Echocardiography Downstream Tasks](https://mlsanity.com/paper/2403.19880/) (2024)
- [Enhancing Efficiency in Vision Transformer Networks: Design Techniques and Insights](https://mlsanity.com/paper/2403.19882/) (2024)
- [Medical Image Segmentation Using Directional Window Attention](https://mlsanity.com/paper/2406.17471/) (2024)
- [Improving 3D Medical Image Segmentation at Boundary Regions using Local Self-attention and Global Volume Mixing](https://mlsanity.com/paper/2410.15360/) (2024)
- [Echo-E3Net: Efficient Endocardial Spatio-Temporal Network for Ejection Fraction Estimation](https://mlsanity.com/paper/2503.17543/) (2025)
- [InceptionMamba: Efficient Multi-Stage Feature Enhancement with Selective State Space Model for Microscopic Medical Image Segmentation](https://mlsanity.com/paper/2506.12208/) (2025)

## Abstract (arXiv)

Microscopic image segmentation is a challenging task, wherein the objective is to assign semantic labels to each pixel in a given microscopic image. While convolutional neural networks (CNNs) form the foundation of many existing frameworks, they often struggle to explicitly capture long-range dependencies. Although transformers were initially devised to address this issue using self-attention, it has been proven that both local and global features are crucial for addressing diverse challenges in microscopic images, including variations in shape, size, appearance, and target region density. In this paper, we introduce SA2-Net, an attention-guided method that leverages multi-scale feature learning to effectively handle diverse structures within microscopic images. Specifically, we propose scale-aware attention (SA2) module designed to capture inherent variations in scales and shapes of microscopic regions, such as cells, for accurate segmentation. This module incorporates local attention at each level of multi-stage features, as well as global attention across multiple resolutions. Furthermore, we address the issue of blurred region boundaries (e.g., cell boundaries) by introducing a novel upsampling strategy called the Adaptive Up-Attention (AuA) module. This module enhances the discriminative ability for improved localization of microscopic regions using an explicit attention mechanism. Extensive experiments on five challenging datasets demonstrate the benefits of our SA2-Net model. Our source code is publicly available at urlhttps://github.com/mustansarfiaz/SA2-Net.