mlsanityData to 7 Oct 2026

Modeling Expert AI Diagnostic Alignment via Immutable Inference Snapshots

arXiv 2602.22973 ↗First posted 26 Feb 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Human-in-the-loop validation is essential in safety-critical clinical AI, yet the transition between initial model inference and expert correction is rarely analyzed as a structured signal. We introduce a diagnostic alignment framework in which the AI-generated image based report is preserved as an immutable inference state and systematically compared with the physician-validated outcome. The inference pipeline integrates a vision-enabled large language model, BERT- based medical entity extraction, and a Sequential Language Model Inference (SLMI) step to enforce domain-consistent refinement prior to expert review. Evaluation on 21 dermatological cases (21 complete AI physician pairs) em- ployed a four-level concordance framework comprising exact primary match rate (PMR), semantic similarity-adjusted rate (AMR), cross-category alignment, and Comprehensive Concordance Rate (CCR). Exact agreement reached 71.4% and remained unchanged under semantic similarity (t = 0.60), while structured cross-category and differential overlap analysis yielded 100% comprehensive concordance (95% CI: [83.9%, 100%]). No cases demonstrated complete diagnostic divergence. These findings show that binary lexical evaluation substantially un- derestimates clinically meaningful alignment. Modeling expert validation as a structured transformation enables signal-aware quantification of correction dynamics and supports traceable, human aligned evaluation of image based clinical decision support systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 12 of its 24 references that are in the map. First posted on arXiv 26 Feb 2026.

Builds on

12 of 24 references.

References · 12

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  4. 2017Deep reinforcement learning from human preferencescited by 3,846
  5. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  6. 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,552
  7. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  8. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  9. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  10. 2025Sequential Diagnosis with Language Modelscited by 53
  11. 2024Dermacen Analytica: A Novel Methodology Integrating Multi-Modal Large Language Models with Machine Learning in tele-dermatologycited by 4
  12. 2024COGNET-MD, an evaluation framework and dataset for Large Language Model benchmarks in the medical domaincited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Data Alignment for Zero-Shot Concept Generation in Dermatology AI
  2. 2026MedLayBench-V: A Large-Scale Benchmark for Expert-Lay Semantic Alignment in Medical Vision Language Models
  3. 2026Mind the Rarities: Can Rare Skin Diseases Be Reliably Diagnosed via Diagnostic Reasoning?
  4. 2025Architecting Clinical Collaboration: Multi-Agent Reasoning Systems for Multimodal Medical VQA
  5. 2025A Medical Multimodal Diagnostic Framework Integrating Vision-Language Models and Logic Tree Reasoning
  6. 2025DiagnosisArena: Benchmarking Diagnostic Reasoning for Large Language Models
  7. 2026MedReaMM: Evaluating Large Multimodal Models on Expert-Level Clinical Diagnostic Synthesis
  8. 2026Clinician input steers AI toward accurate and harmful recommendations