# LesionAid: Vision Transformers-based Skin Lesion Generation and Classification

arXiv: https://arxiv.org/abs/2302.01104 · HTML: https://mlsanity.com/paper/2302.01104/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 14 references that are in the map.
- First posted on arXiv 2 Feb 2023.

## Builds on (most cited first, 1 of 1)

- [Skin Lesion Classification Using Hybrid Deep Neural Networks](https://mlsanity.com/paper/1702.08434/) (2017)

## Built on by (most cited first, 6 of 6)

- [A Recent Survey of Vision Transformers for Medical Image Segmentation](https://mlsanity.com/paper/2312.00634/) (2023)
- [Breast Cancer Segmentation using Attention-based Convolutional Network and Explainable AI](https://mlsanity.com/paper/2305.14389/) (2023)
- [When Medical Imaging Met Self-Attention: A Love Story That Didn't Quite Work Out](https://mlsanity.com/paper/2404.12295/) (2024)
- [Enhancing Skin Disease Classification Leveraging Transformer-based Deep Learning Architectures and Explainable AI](https://mlsanity.com/paper/2407.14757/) (2024)
- [Deep Learning-based Spatio Temporal Facial Feature Visual Speech Recognition](https://mlsanity.com/paper/2305.00552/) (2023)
- [CoVid-19 Detection leveraging Vision Transformers and Explainable AI](https://mlsanity.com/paper/2307.16033/) (2023)

## Abstract (arXiv)

Skin cancer is one of the most prevalent forms of human cancer. It is recognized mainly visually, beginning with clinical screening and continuing with the dermoscopic examination, histological assessment, and specimen collection. Deep convolutional neural networks (CNNs) perform highly segregated and potentially universal tasks against a classified finegrained object. This research proposes a novel multi-class prediction framework that classifies skin lesions based on ViT and ViTGAN. Vision transformers-based GANs (Generative Adversarial Networks) are utilized to tackle the class imbalance. The framework consists of four main phases: ViTGANs, Image processing, and explainable AI. Phase 1 consists of generating synthetic images to balance all the classes in the dataset. Phase 2 consists of applying different data augmentation techniques and morphological operations to increase the size of the data. Phases 3 & 4 involve developing a ViT model for edge computing systems that can identify patterns and categorize skin lesions from the user's skin visible in the image. In phase 3, after classifying the lesions into the desired class with ViT, we will use explainable AI (XAI) that leads to more explainable results (using activation maps, etc.) while ensuring high predictive accuracy. Real-time images of skin diseases can capture by a doctor or a patient using the camera of a mobile application to perform an early examination and determine the cause of the skin lesion. The whole framework is compared with the existing frameworks for skin lesion detection.