mlsanityData to 14 Oct 2026

Enhancing Trustworthy Clinical Diagnosis Decision-Making in Large Language Models via Etiology-Aware Attention Supervision

arXiv 2508.00285 ↗First posted 1 Aug 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Objective: Large Language Models (LLMs) have demonstrated strong capabilities in medical text understanding and generation. However, their trustworthiness in diagnosis-oriented medical tasks remains constrained by the lack of structured guidance on how clinically relevant diagnostic evidence is internally attended to and utilized during model learning. Method: We propose an Etiology-Aware Attention Supervision framework that introduces structured etiological information as an external supervisory signal for training large language models. Specifically, we construct Clinical Etiology Schema (CES) derived from authoritative clinical guidelines for three acute abdominal conditions: acute appendicitis, acute pancreatitis, and acute cholecystitis. Based on CES annotations, we develop an Etiology-Aware Head Identification strategy to identify attention heads that consistently align with etiological evidence. Building on this analysis, we design a structure-guided parameter-efficient fine-tuning approach that steers attention distributions toward clinically relevant evidence through an additional supervision loss, without modifying the base model architecture. Result: Experiments conducted on a Consistent Diagnosis Cohort demonstrate that the proposed framework improves average diagnostic accuracy by 15.65% compared with baseline models. Attention-based metrics, including Inference Focus Score and Inference Attention Frequency, show more concentrated attention on etiologically relevant evidence. External evaluation on a Discrepant Diagnosis Cohort further confirms the robustness of diagnostic performance improvements under real-world clinical inconsistencies.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 40 references that are in the map. First posted on arXiv 1 Aug 2025.

Builds on

15 of 40 references.

References · 15

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2024Qwen2.5 Technical Reportcited by 4,942
  4. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  5. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  6. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  7. 2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Headscited by 288
  8. 2024Retrieval Head Mechanistically Explains Long-Context Factualitycited by 232
  9. 2025MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphscited by 110
  10. 2023Diagnostic Reasoning Prompts Reveal the Potential for Large Language Model Interpretability in Medicinecited by 83
  11. 2024Attention Heads of Large Language Models: A Surveycited by 67
  12. 2023Large Language Models are Clinical Reasoners: Reasoning-Aware Diagnosis Framework with Prompt-Generated Rationalescited by 33
3 more
  1. 2025Spotlight Your Instructions: Instruction-following with Dynamic Attention Steeringcited by 21
  2. 2024Guiding Clinical Reasoning with Large Language Models via Knowledge Seedscited by 19
  3. 2025Improving Contextual Faithfulness of Large Language Models via Retrieval Heads-Induced Optimizationcited by 17

Built on by

Not cited yet.