# NAM: Normalization-based Attention Module

arXiv: https://arxiv.org/abs/2111.12419 · HTML: https://mlsanity.com/paper/2111.12419/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 11 references that are in the map.
- First posted on arXiv 24 Nov 2021.

## Builds on (most cited first, 8 of 8)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Learning Efficient Convolutional Networks through Network Slimming](https://mlsanity.com/paper/1708.06519/) (2017)
- [Residual Attention Network for Image Classification](https://mlsanity.com/paper/1704.06904/) (2017)
- [Relation-Aware Global Attention for Person Re-identification](https://mlsanity.com/paper/1904.02998/) (2019)
- [Rotate to Attend: Convolutional Triplet Attention Module](https://mlsanity.com/paper/2010.03045/) (2020)
- [Global Second-order Pooling Convolutional Networks](https://mlsanity.com/paper/1811.12006/) (2018)

## Built on by (most cited first, 8 of 8)

- [Efficient Multi-Scale Attention Module with Cross-Spatial Learning](https://mlsanity.com/paper/2305.13563/) (2023)
- [MobileIE: An Extremely Lightweight and Effective ConvNet for Real-Time Image Enhancement on Mobile Devices](https://mlsanity.com/paper/2507.01838/) (2025)
- [RS-YOLOX: A High Precision Detector for Object Detection in Satellite Remote Sensing Images](https://mlsanity.com/paper/2502.02850/) (2025)
- [Cross-modal Local Shortest Path and Global Enhancement for Visible-Thermal Person Re-Identification](https://mlsanity.com/paper/2206.04401/) (2022)
- [Analysis of Object Detection Models for Tiny Object in Satellite Imagery: A Dataset-Centric Approach](https://mlsanity.com/paper/2412.10453/) (2024)
- [Finger Multimodal Feature Fusion and Recognition Based on Channel Spatial Attention](https://mlsanity.com/paper/2209.02368/) (2022)
- [Intensive Vision-guided Network for Radiology Report Generation](https://mlsanity.com/paper/2402.03754/) (2024)
- [Feature Recalibration Based Olfactory-Visual Multimodal Model for Enhanced Rice Deterioration Detection](https://mlsanity.com/paper/2602.14408/) (2026)

## Abstract (arXiv)

Recognizing less salient features is the key for model compression. However, it has not been investigated in the revolutionary attention mechanisms. In this work, we propose a novel normalization-based attention module (NAM), which suppresses less salient weights. It applies a weight sparsity penalty to the attention modules, thus, making them more computational efficient while retaining similar performance. A comparison with three other attention mechanisms on both Resnet and Mobilenet indicates that our method results in higher accuracy. Code for this paper can be publicly accessed at https://github.com/Christian-lyc/NAM.