# Class Attention to Regions of Lesion for Imbalanced Medical Image Recognition

arXiv: https://arxiv.org/abs/2307.10036 · HTML: https://mlsanity.com/paper/2307.10036/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 57 references that are in the map.
- First posted on arXiv 19 Jul 2023.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss](https://mlsanity.com/paper/1906.07413/) (2019)
- [Decoupling Representation and Classifier for Long-Tailed Recognition](https://mlsanity.com/paper/1910.09217/) (2019)
- [Learning to Reweight Examples for Robust Deep Learning](https://mlsanity.com/paper/1803.09050/) (2018)
- [Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1710.05006/) (2017)
- [Stand-Alone Self-Attention in Vision Models](https://mlsanity.com/paper/1906.05909/) (2019)
- [The Effectiveness of Data Augmentation in Image Classification using Deep Learning](https://mlsanity.com/paper/1712.04621/) (2017)
- [SCA-CNN: Spatial and Channel-wise Attention in Convolutional Networks for Image Captioning](https://mlsanity.com/paper/1611.05594/) (2016)
- [Cost Sensitive Learning of Deep Feature Representations from Imbalanced Data](https://mlsanity.com/paper/1508.03422/) (2015)
- [Medical Image Analysis using Convolutional Neural Networks: A Review](https://mlsanity.com/paper/1709.02250/) (2017)
- [Multi-label Thoracic Disease Image Classification with Cross-Attention Networks](https://mlsanity.com/paper/2007.10859/) (2020)
- [Fully Convolutional Network Ensembles for White Matter Hyperintensities Segmentation in MR Images](https://mlsanity.com/paper/1802.05203/) (2018)
- [ChestNet: A Deep Neural Network for Classification of Thoracic Diseases on Chest Radiography](https://mlsanity.com/paper/1807.03058/) (2018)
- [HR-CAM: Precise Localization of Pathology Using Multi-level Learning in CNNs](https://mlsanity.com/paper/1909.12919/) (2019)
- [Multi Scale Curriculum CNN for Context-Aware Breast MRI Malignancy Classification](https://mlsanity.com/paper/1906.06058/) (2019)

## Built on by (most cited first, 3 of 3)

- [MiM: Mask in Mask Self-Supervised Pre-Training for 3D Medical Image Analysis](https://mlsanity.com/paper/2404.15580/) (2024)
- [Decentralized LoRA augmented transformer with multi-scale feature learning for secured eye diagnosis](https://mlsanity.com/paper/2505.06982/) (2025)
- [Class-Specific Branch Attention for Mitigating Gradient Interference under Class Imbalance](https://mlsanity.com/paper/2606.05740/) (2026)

## Abstract (arXiv)

Automated medical image classification is the key component in intelligent diagnosis systems. However, most medical image datasets contain plenty of samples of common diseases and just a handful of rare ones, leading to major class imbalances. Currently, it is an open problem in intelligent diagnosis to effectively learn from imbalanced training data. In this paper, we propose a simple yet effective framework, named Class Attention to REgions of the lesion (CARE), to handle data imbalance issues by embedding attention into the training process of Convolutional Neural Networks (CNNs). The proposed attention module helps CNNs attend to lesion regions of rare diseases, therefore helping CNNs to learn their characteristics more effectively. In addition, this attention module works only during the training phase and does not change the architecture of the original network, so it can be directly combined with any existing CNN architecture. The CARE framework needs bounding boxes to represent the lesion regions of rare diseases. To alleviate the need for manual annotation, we further developed variants of CARE by leveraging the traditional saliency methods or a pretrained segmentation model for bounding box generation. Results show that the CARE variants with automated bounding box generation are comparable to the original CARE framework with textitmanual bounding box annotations. A series of experiments on an imbalanced skin image dataset and a pneumonia dataset indicates that our method can effectively help the network focus on the lesion regions of rare diseases and remarkably improves the classification performance of rare diseases.