mlsanityData to 14 Oct 2026

Evaluating Deep Neural Networks Trained on Clinical Images in Dermatology with the Fitzpatrick 17k Dataset

arXiv 2104.09957 ↗First posted 20 Apr 2021cs.CVcited by 185

Not on the map. Why?

Cited by 20 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

How does the accuracy of deep neural network models trained to classify clinical images of skin conditions vary across skin color? While recent studies demonstrate computer vision models can serve as a useful decision support tool in healthcare and provide dermatologist-level classification on a number of specific tasks, darker skin is underrepresented in the data. Most publicly available data sets do not include Fitzpatrick skin type labels. We annotate 16,577 clinical images sourced from two dermatology atlases with Fitzpatrick skin type labels and open-source these annotations. Based on these labels, we find that there are significantly more images of light skin types than dark skin types in this dataset. We train a deep neural network model to classify 114 skin conditions and find that the model is most accurate on skin types similar to those it was trained on. In addition, we evaluate how an algorithmic approach to identifying skin tones, individual typology angle, compares with Fitzpatrick skin type labels annotated by a team of human labelers.

Key facts. Cited by 185 papers in the map (data through 14 Oct 2026). Builds on 15 of its 58 references that are in the map. First posted on arXiv 20 Apr 2021.

Builds on

15 of 58 references.

References · 15

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  2. 2019Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)cited by 669
  3. 2017Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)cited by 549
  4. 2021Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarkscited by 395
  5. 2019Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imagingcited by 216
  6. 2020A Patient-Centric Dataset of Images and Metadata for Identifying Melanomas Using Clinical Contextcited by 146
  7. 2018Why Is My Classifier Discriminatory?cited by 143
  8. 2019A deep learning system for differential diagnosis of skin diseasescited by 118
  9. 2020PAD-UFES-20: a skin lesion dataset composed of patient data and clinical images collected from smartphonescited by 118
  10. 2019Diversity in Facescited by 77
  11. 2020Debiasing Skin Lesion Datasets and Models? Not So Fastcited by 43
  12. 2019Estimating Skin Tone and Effects on Classification Performance in Dermatology Datasetscited by 29
3 more
  1. 2020Risk of Training Diagnostic Algorithms on Data with Demographic Biascited by 23
  2. 2017Human Skin Detection Using RGB, HSV and YCbCr Color Modelscited by 16
  3. 2018Deep-Learning Ensembles for Skin-Lesion Segmentation, Analysis, Classification: RECOD Titans at ISIC Challenge 2018cited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 20; 2023: 37; 2024: 39; 2025: 46; 2026: 39.

185 papers.

Sort:

By area:

  1. 2022Post-hoc Concept Bottleneck Modelscited by 293
  2. 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoningcited by 230
  3. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLMcited by 184
  4. 2021Disparities in Dermatology AI: Assessments Using Diverse Clinical Imagescited by 134
  5. 2021Fair Conformal Predictors for Applications in Medical Imagingcited by 74
  6. 2024FairCLIP: Harnessing Fairness in Vision-Language Learningcited by 69
  7. 2022MEDFAIR: Benchmarking Fairness for Medical Imagingcited by 68
  8. 2023A Closer Look at the Intervention Procedure of Concept Bottleneck Modelscited by 64
  9. 2022FairPrune: Achieving Fairness Through Pruning for Dermatological Disease Diagnosiscited by 59
  10. 2023Parameter-Efficient Fine-Tuning for Medical Image Analysis: The Missed Opportunitycited by 56
  11. 2021Meaningfully Debugging Model Mistakes using Conceptual Counterfactual Explanationscited by 53
  12. 2023Deep Learning Approaches for Data Augmentation in Medical Imaging: A Reviewcited by 48
38 more
  1. 2023Federated Conformal Predictors for Distributed Uncertainty Quantificationcited by 46
  2. 2023SkinCon: A skin disease dataset densely annotated by domain experts for fine-grained model debugging and analysiscited by 44
  3. 2022Towards Transparency in Dermatology Image Datasets with Skin Tone Annotations by Experts, Crowds, and an Algorithmcited by 35
  4. 2022Explainable Deep Learning Methods in Medical Image Classification: A Surveycited by 32
  5. 2022VeriX: Towards Verified Explainability of Deep Neural Networkscited by 30
  6. 2022Addressing Fairness Issues in Deep Learning-Based Medical Image Analysis: A Systematic Reviewcited by 28
  7. 2025Derm1M: A Million-scale Vision-Language Dataset Aligned with Clinical Ontology Knowledge for Dermatologycited by 28
  8. 2023Towards Trustable Skin Cancer Diagnosis via Rewriting Model's Decisioncited by 27
  9. 2022Improving dermatology classifiers across populations using images generated by large diffusion modelscited by 25
  10. 2023Beyond Confidence: Reliable Models Should Also Consider Atypicalitycited by 25
  11. 2022Detecting Melanoma Fairly: Skin Tone Detection and Debiasing for Skin Lesion Classificationcited by 23
  12. 2023FairTune: Optimizing Parameter Efficient Fine Tuning for Fairness in Medical Image Analysiscited by 23
  13. 2022A Survey on Deep Learning for Skin Lesion Segmentationcited by 22
  14. 2022FairDisCo: Fairer AI in Dermatology via Disentanglement Contrastive Learningcited by 22
  15. 2023Consensus and Subjectivity of Skin Tone Annotation for ML Fairnesscited by 22
  16. 2023Augmenting medical image classifiers with synthetic data from latent diffusion modelscited by 22
  17. 2023FairAdaBN: Mitigating unfairness with adaptive batch normalization and its application to dermatological disease classificationcited by 19
  18. 2024FairDomain: Achieving Fairness in Cross-Domain Medical Image Segmentation and Classificationcited by 19
  19. 2024Self-eXplainable AI for Medical Image Analysis: A Survey and New Outlookscited by 18
  20. 2022CIRCLe: Color Invariant Representation Learning for Unbiased Classification of Skin Lesionscited by 17
  21. 2024Integrating Clinical Knowledge into Concept Bottleneck Modelscited by 17
  22. 2021Analysis of Manual and Automated Skin Tone Assignments for Face Recognition Applicationscited by 16
  23. 20253MDBench: Medical Multimodal Multi-agent Dialogue Benchmarkcited by 16
  24. 2023Robust T-Loss for Medical Image Segmentationcited by 14
  25. 2023Toward Fairness Through Fair Multi-Exit Framework for Dermatological Disease Diagnosiscited by 14
  26. 2025MM-Skin: Enhancing Dermatology Vision-Language Model with an Image-Text Dataset Derived from Textbookscited by 14
  27. 2023FairVision: Equitable Deep Learning for Eye Disease Screening via Fair Identity Scalingcited by 13
  28. 2024Utility-Directed Conformal Prediction: A Decision-Aware Framework for Actionable Uncertainty Quantificationcited by 13
  29. 2024BiasPruner: Debiased Continual Learning for Medical Image Classificationcited by 12
  30. 2024OxonFair: A Flexible Toolkit for Algorithmic Fairnesscited by 12
  31. 2025MAKE: Multi-Aspect Knowledge-Enhanced Vision-Language Pretraining for Zero-shot Dermatological Assessmentcited by 12
  32. 2023Skin Deep: Investigating Subjectivity in Skin Tone Annotations for Computer Vision Benchmark Datasetscited by 11
  33. 2023The Role of Subgroup Separability in Group-Fair Medical Image Classificationcited by 11
  34. 2024Concept-Attention Whitening for Interpretable Skin Lesion Diagnosiscited by 11
  35. 2023Estimating Uncertainty in Multimodal Foundation Models using Public Internet Datacited by 10
  36. 2024Achieve Fairness without Demographics for Dermatological Disease Diagnosiscited by 10
  37. 2024DERM12345: A Large, Multisource Dermatoscopic Skin Lesion Dataset with 38 Subclassescited by 10
  38. 2023Achieving Reliable and Fair Skin Lesion Diagnosis via Unsupervised Domain Adaptationcited by 9

50 most cited of 185 All 185 (JSON)