# Skin Lesion Classification Using CNNs with Patch-Based Attention and Diagnosis-Guided Loss Weighting

arXiv: https://arxiv.org/abs/1905.02793 · HTML: https://mlsanity.com/paper/1905.02793/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 May 2019.

## Built on by (most cited first, 10 of 10)

- [Single Model Deep Learning on Imbalanced Small Datasets for Skin Lesion Classification](https://mlsanity.com/paper/2102.01284/) (2021)
- [FairDisCo: Fairer AI in Dermatology via Disentanglement Contrastive Learning](https://mlsanity.com/paper/2208.10013/) (2022)
- [A survey, review, and future trends of skin lesion segmentation and classification](https://mlsanity.com/paper/2208.12232/) (2022)
- [Knowledge-aware Deep Framework for Collaborative Skin Lesion Segmentation and Melanoma Recognition](https://mlsanity.com/paper/2106.03455/) (2021)
- [Attention mechanisms and deep learning for machine vision: A survey of the state of the art](https://mlsanity.com/paper/2106.07550/) (2021)
- [Deep Clustering via Center-Oriented Margin Free-Triplet Loss for Skin Lesion Detection in Highly Imbalanced Datasets](https://mlsanity.com/paper/2204.02275/) (2022)
- [AGMR-Net: Attention Guided Multiscale Recovery framework for stroke segmentation](https://mlsanity.com/paper/2202.13687/) (2022)
- [Multi-task Explainable Skin Lesion Classification](https://mlsanity.com/paper/2310.07209/) (2023)
- [Progressive Class-Wise Attention (PCA) Approach for Diagnosing Skin Lesions](https://mlsanity.com/paper/2306.07300/) (2023)
- [Towards Facilitated Fairness Assessment of AI-based Skin Lesion Classifiers Through GenAI-based Image Synthesis](https://mlsanity.com/paper/2507.17860/) (2025)

## Abstract (arXiv)

Objective: This work addresses two key problems of skin lesion classification. The first problem is the effective use of high-resolution images with pretrained standard architectures for image classification. The second problem is the high class imbalance encountered in real-world multi-class datasets. Methods: To use high-resolution images, we propose a novel patch-based attention architecture that provides global context between small, high-resolution patches. We modify three pretrained architectures and study the performance of patch-based attention. To counter class imbalance problems, we compare oversampling, balanced batch sampling, and class-specific loss weighting. Additionally, we propose a novel diagnosis-guided loss weighting method which takes the method used for ground-truth annotation into account. Results: Our patch-based attention mechanism outperforms previous methods and improves the mean sensitivity by 7%. Class balancing significantly improves the mean sensitivity and we show that our diagnosis-guided loss weighting method improves the mean sensitivity by 3% over normal loss balancing. Conclusion: The novel patch-based attention mechanism can be integrated into pretrained architectures and provides global context between local patches while outperforming other patch-based methods. Hence, pretrained architectures can be readily used with high-resolution images without downsampling. The new diagnosis-guided loss weighting method outperforms other methods and allows for effective training when facing class imbalance. Significance: The proposed methods improve automatic skin lesion classification. They can be extended to other clinical applications where high-resolution image data and class imbalance are relevant.