mlsanityData to 14 Oct 2026

Exploring Advances in Transformers and CNN for Skin Lesion Diagnosis on Small Datasets

arXiv 2205.15442 ↗First posted 30 May 2022cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Skin cancer is one of the most common types of cancer in the world. Different computer-aided diagnosis systems have been proposed to tackle skin lesion diagnosis, most of them based in deep convolutional neural networks. However, recent advances in computer vision achieved state-of-art results in many tasks, notably Transformer-based networks. We explore and evaluate advances in computer vision architectures, training methods and multimodal feature fusion for skin lesion diagnosis task. Experiments show that PiT (0.800 ± 0.006), CoaT (0.780 ± 0.024) and ViT (0.771 ± 0.018) backbone models with MetaBlock fusion achieved state-of-art results for balanced accuracy metric in PAD-UFES-20 dataset.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 43 of its 56 references that are in the map. First posted on arXiv 30 May 2022.

Builds on

43 of 56 references.

References · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  7. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  8. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  9. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  10. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  11. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  12. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,432
31 more
  1. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,311
  2. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,228
  3. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,135
  4. 2021Transformers in Vision: A Surveycited by 1,013
  5. 2020Designing Network Design Spacescited by 960
  6. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 903
  7. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  8. 2019Big Transfer (BiT): General Visual Representation Learningcited by 769
  9. 2021Going deeper with Image Transformerscited by 764
  10. 2021Transformer in Transformercited by 633
  11. 2019ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networkscited by 575
  12. 2021How to train your ViT? Data, Augmentation, and Regularization in Vision Transformerscited by 569
  13. 2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565
  14. 2021Intriguing Properties of Vision Transformerscited by 446
  15. 2021Bottleneck Transformers for Visual Recognitioncited by 364
  16. 2021Rethinking Spatial Dimensions of Vision Transformerscited by 361
  17. 2021High-Performance Large-Scale Image Recognition Without Normalizationcited by 360
  18. 2021XCiT: Cross-Covariance Image Transformerscited by 338
  19. 2020Are we done with ImageNet?cited by 333
  20. 2019GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyondcited by 331
  21. 2021Knowledge distillation: A good teacher is patient and consistentcited by 259
  22. 2019Billion-scale semi-supervised learning for image classificationcited by 232
  23. 2021Co-Scale Conv-Attentional Image Transformerscited by 224
  24. 2021Scaling Local Self-Attention for Parameter Efficient Visual Backbonescited by 220
  25. 2020PAD-UFES-20: a skin lesion dataset composed of patient data and clinical images collected from smartphonescited by 118
  26. 2020Deep Multimodal Fusion by Channel Exchangingcited by 118
  27. 2021A Survey of Visual Transformerscited by 117
  28. 2021Fast and Accurate Model Scalingcited by 46
  29. 2020Rethinking Channel Dimensions for Efficient Model Designcited by 41
  30. 2019The impact of patient clinical information on automated skin cancer detectioncited by 21
  31. 2021Improved skin lesion recognition by a Self-Supervised Curricular Deep Learning approachcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 2; 2026: 1.

4 papers.

  1. 2023Towards Generalist Biomedical AIcited by 289
  2. 2025Diffusion models applied to skin and oral cancer classificationcited by 3
  3. 2025Skewness-Guided Pruning of Multimodal Swin Transformers for Federated Skin Lesion Classification on Edge Devicescited by 1
  4. 2026An approach with Visual and Tabular Mamba to multimodal medical data using Mixed Fusioncited by 0