# Learn To Pay Attention

arXiv: https://arxiv.org/abs/1804.02391 · HTML: https://mlsanity.com/paper/1804.02391/

## Key facts

- Cited by 83 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Apr 2018.

## Built on by (most cited first, 30 of 83)

- [Explainable artificial intelligence (XAI) in deep learning-based medical image analysis](https://mlsanity.com/paper/2107.10912/) (2021)
- [A Comprehensive Survey of Scene Graphs: Generation and Application](https://mlsanity.com/paper/2104.01111/) (2021)
- [Recent advances and clinical applications of deep learning in medical image analysis](https://mlsanity.com/paper/2105.13381/) (2021)
- [Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methods](https://mlsanity.com/paper/2111.02398/) (2021)
- [Deep Learning for Survival Analysis: A Review](https://mlsanity.com/paper/2305.14961/) (2023)
- [Sub-word Level Lip Reading With Visual Attention](https://mlsanity.com/paper/2110.07603/) (2021)
- [Visual Navigation with Spatial Attention](https://mlsanity.com/paper/2104.09807/) (2021)
- [A Survey on Intrinsic Images: Delving Deep Into Lambert and Beyond](https://mlsanity.com/paper/2112.03842/) (2021)
- [Visual Attention Methods in Deep Learning: An In-Depth Survey](https://mlsanity.com/paper/2204.07756/) (2022)
- [RAMS-Trans: Recurrent Attention Multi-scale Transformer forFine-grained Image Recognition](https://mlsanity.com/paper/2107.08192/) (2021)
- [Mediastinal lymph nodes segmentation using 3D convolutional neural network ensembles and anatomical priors guiding](https://mlsanity.com/paper/2102.06515/) (2021)
- [Self-eXplainable AI for Medical Image Analysis: A Survey and New Outlooks](https://mlsanity.com/paper/2410.02331/) (2024)
- [Sequential vessel segmentation via deep channel attention network](https://mlsanity.com/paper/2102.05229/) (2021)
- [CAT-Net: A Cross-Slice Attention Transformer Model for Prostate Zonal Segmentation in MRI](https://mlsanity.com/paper/2203.15163/) (2022)
- [A survey on attention mechanisms for medical applications: are we moving towards better algorithms?](https://mlsanity.com/paper/2204.12406/) (2022)
- [Attention Aware Wavelet-based Detection of Morphed Face Images](https://mlsanity.com/paper/2106.15686/) (2021)
- [Spatial Mixture-of-Experts](https://mlsanity.com/paper/2211.13491/) (2022)
- [Revisiting Near/Remote Sensing with Geospatial Attention](https://mlsanity.com/paper/2204.01807/) (2022)
- [Passive Attention in Artificial Neural Networks Predicts Human Visual Selectivity](https://mlsanity.com/paper/2107.07013/) (2021)
- [Meningioma segmentation in T1-weighted MRI leveraging global context and attention mechanisms](https://mlsanity.com/paper/2101.07715/) (2021)
- [E-PixelHop: An Enhanced PixelHop Method for Object Classification](https://mlsanity.com/paper/2107.02966/) (2021)
- [Deep Learning Generates Synthetic Cancer Histology for Explainability and Education](https://mlsanity.com/paper/2211.06522/) (2022)
- [Towards Interpretable Solar Flare Prediction with Attention-based Deep Neural Networks](https://mlsanity.com/paper/2309.04558/) (2023)
- [Few-shot Learning for Slot Tagging with Attentive Relational Network](https://mlsanity.com/paper/2103.02333/) (2021)
- [Multi-layered Semantic Representation Network for Multi-label Image Classification](https://mlsanity.com/paper/2106.11596/) (2021)
- [Information Bottleneck Approach to Spatial Attention Learning](https://mlsanity.com/paper/2108.03418/) (2021)
- [Placenta Segmentation in Ultrasound Imaging: Addressing Sources of Uncertainty and Limited Field-of-View](https://mlsanity.com/paper/2206.14746/) (2022)
- [Improved Kidney Stone Recognition Through Attention and Multi-View Feature Fusion Strategies](https://mlsanity.com/paper/2211.02967/) (2022)
- [TAME: Attention Mechanism Based Feature Fusion for Generating Explanation Maps of Convolutional Neural Networks](https://mlsanity.com/paper/2301.07407/) (2023)
- [CONFINE: Conformal Prediction for Interpretable Neural Networks](https://mlsanity.com/paper/2406.00539/) (2024)

## Abstract (arXiv)

We propose an end-to-end-trainable attention module for convolutional neural network (CNN) architectures built for image classification. The module takes as input the 2D feature vector maps which form the intermediate representations of the input image at different stages in the CNN pipeline, and outputs a 2D matrix of scores for each map. Standard CNN architectures are modified through the incorporation of this module, and trained under the constraint that a convex combination of the intermediate 2D feature vectors, as parameterised by the score matrices, must textitalone be used for classification. Incentivised to amplify the relevant and suppress the irrelevant or misleading, the scores thus assume the role of attention values. Our experimental observations provide clear evidence to this effect: the learned attention maps neatly highlight the regions of interest while suppressing background clutter. Consequently, the proposed function is able to bootstrap standard CNN architectures for the task of image classification, demonstrating superior generalisation over 6 unseen benchmark datasets. When binarised, our attention maps outperform other CNN-based attention maps, traditional saliency maps, and top object proposals for weakly supervised segmentation as demonstrated on the Object Discovery dataset. We also demonstrate improved robustness against the fast gradient sign method of adversarial attack.