# DMSANet: Dual Multi Scale Attention Network

arXiv: https://arxiv.org/abs/2106.08382 · HTML: https://mlsanity.com/paper/2106.08382/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 34 references that are in the map.
- First posted on arXiv 13 Jun 2021.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1910.03151/) (2019)
- [Res2Net: A New Multi-scale Backbone Architecture](https://mlsanity.com/paper/1904.01169/) (2019)
- [ResNeSt: Split-Attention Networks](https://mlsanity.com/paper/2004.08955/) (2020)
- [Selective Kernel Networks](https://mlsanity.com/paper/1903.06586/) (2019)
- [Residual Attention Network for Image Classification](https://mlsanity.com/paper/1704.06904/) (2017)
- [Attention Augmented Convolutional Networks](https://mlsanity.com/paper/1904.09925/) (2019)
- [GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond](https://mlsanity.com/paper/1904.11492/) (2019)
- [Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks](https://mlsanity.com/paper/1810.12348/) (2018)
- [Attention Branch Network: Learning of Attention Mechanism for Visual Explanation](https://mlsanity.com/paper/1812.10025/) (2018)
- [Squeeze-and-Attention Networks for Semantic Segmentation](https://mlsanity.com/paper/1909.03402/) (2019)
- [SA-Net: Shuffle Attention for Deep Convolutional Neural Networks](https://mlsanity.com/paper/2102.00240/) (2021)
- [Pyramidal Convolution: Rethinking Convolutional Neural Networks for Visual Recognition](https://mlsanity.com/paper/2006.11538/) (2020)
- [ChannelNets: Compact and Efficient Convolutional Neural Networks via Channel-Wise Convolutions](https://mlsanity.com/paper/1809.01330/) (2018)
- [Monocular Depth Estimation Using Multi Scale Neural Network And Feature Fusion](https://mlsanity.com/paper/2009.09934/) (2020)
- [Semantic Segmentation With Multi Scale Spatial Attention For Self Driving Cars](https://mlsanity.com/paper/2007.12685/) (2020)
- [Bayesian Multi-Scale Neural Network for Crowd Counting](https://mlsanity.com/paper/2007.14245/) (2020)

## Built on by (most cited first, 8 of 8)

- [Modality-agnostic Domain Generalizable Medical Image Segmentation by Multi-Frequency in Multi-Scale Attention](https://mlsanity.com/paper/2405.06284/) (2024)
- [Multi-scale Transformer Network with Edge-aware Pre-training for Cross-Modality MR Image Synthesis](https://mlsanity.com/paper/2212.01108/) (2022)
- [LOGCAN++: Adaptive Local-global class-aware network for semantic segmentation of remote sensing imagery](https://mlsanity.com/paper/2406.16502/) (2024)
- [ViTBIS: Vision Transformer for Biomedical Image Segmentation](https://mlsanity.com/paper/2201.05920/) (2022)
- [LoG-CAN: local-global Class-aware Network for semantic segmentation of remote sensing images](https://mlsanity.com/paper/2303.07747/) (2023)
- [AA3DNet: Attention Augmented Real Time 3D Object Detection](https://mlsanity.com/paper/2107.12137/) (2021)
- [AASeg: Attention Aware Network for Real Time Semantic Segmentation](https://mlsanity.com/paper/2108.04349/) (2021)
- [Superresolution and Segmentation of OCT scans using Multi-Stage adversarial Guided Attention Training](https://mlsanity.com/paper/2206.05277/) (2022)

## Abstract (arXiv)

Attention mechanism of late has been quite popular in the computer vision community. A lot of work has been done to improve the performance of the network, although almost always it results in increased computational complexity. In this paper, we propose a new attention module that not only achieves the best performance but also has lesser parameters compared to most existing models. Our attention module can easily be integrated with other convolutional neural networks because of its lightweight nature. The proposed network named Dual Multi Scale Attention Network (DMSANet) is comprised of two parts: the first part is used to extract features at various scales and aggregate them, the second part uses spatial and channel attention modules in parallel to adaptively integrate local features with their global dependencies. We benchmark our network performance for Image Classification on ImageNet dataset, Object Detection and Instance Segmentation both on MS COCO dataset.