mlsanityData to 14 Oct 2026

When LLMs Analyze Scars: From Images to Clinically-Meaningful Features

arXiv 2606.18063 ↗First posted 16 Jun 2026cs.CV cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Medical image classification faces a fundamental dilemma: while deep learning models achieve remarkable performance at scale, real-world clinical scenarios often suffer from severe data scarcity due to annotation costs, privacy constraints, and disease rarity. This challenge is particularly pronounced in pathological scar classification, where differentiating keloids from hypertrophic scars requires subtle expert knowledge and labeled images are extremely limited. We propose a novel paradigm that repositions large language models (LLMs) as knowledge-driven feature engineers rather than end-to-end classifiers. We call this framework ScaFE (Scar Feature Engineering). Our key insight is that LLMs encode rich medical knowledge that can be externalized as executable feature extraction code, enabling the transformation of high-dimensional images into low-dimensional, clinically interpretable representations. Specifically, we prompt an LLM with established scar assessment criteria to generate deterministic Python code that extracts features aligned with clinical scoring systems such as the Vancouver Scar Scale. Our approach offers three key advantages: (1) data efficiency, achieving robust performance with limited training samples by decoupling knowledge acquisition from statistical learning; (2) privacy preservation, as raw images are processed locally without exposure to external LLMs; and (3) interpretability, through explicit features grounded in clinical reasoning. Extensive experiments on scar classification demonstrate that our method consistently outperforms end-to-end deep learning baselines or using LLMs as black-box classifiers under limited data conditions, establishing a promising direction for integrating LLMs into data-efficient and clinically transparent medical AI systems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 28 references that are in the map. First posted on arXiv 16 Jun 2026.

Builds on

10 of 28 references.

References · 10

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  4. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  5. 2017A Survey on Deep Learning in Medical Image Analysiscited by 1,186
  6. 2020Concept Bottleneck Modelscited by 1,032
  7. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
  8. 2020Generating Radiology Reports via Memory-driven Transformercited by 368
  9. 2020Neurosymbolic AI: The 3rd Wavecited by 255
  10. 2024PTaRL: Prototype-based Tabular Representation Learning via Space Calibrationcited by 28

Built on by

Not cited yet.