mlsanityData to 14 Oct 2026

Clinical Concept Centers in LLMs

arXiv 2610.02829 ↗First posted 2 Oct 2026cs.CL cs.LGcited by 0

On the map: Mechanistic interpretability → Knowledge and representations inside LLMs

Abstract (arXiv)

Large language models are increasingly used in clinical settings. However, research into the reliability and performance of these models has focused almost entirely on the language substrate, scoring what the model says. Mechanistic interpretability has found that the latent space carries a higher fidelity of representation than the text: internal representations not only encode substantially more than the output verbalizes, but the stated reasoning also systematically omits features that causally drive the answer. An evaluation of model behavior in terms of mechanistic interpretability has not been explored in clinical decision support. In this work, we extend behavioral evaluation into the latent space and ask whether clinical concepts exist as locatable, causally used representations inside open-weight LLMs. We find dedicated clinical concept centers in the latent space of all eleven open models we test. These concept centers are interpretable, firing only on their aligned clinical narratives, and meaningfully and causally drive model behavior in both constrained and open-ended settings. They are not just analytical representations, but circuits that can be utilized in clinical practice, and we explore their use from the perspective of both evaluation and performance. From the evaluation standpoint, models stay internally coherent and keep using the relevant concept centers even under adversarial role-based priming, while aligned priming improves downstream clinical performance. From a performance perspective, we simulate realistic deployment settings and find that steering models along these centers leads to meaningful downstream improvements. Finally, we conduct a blinded clinician validation and find the activation and usage of these concept centers predicts clinicians preferences.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Knowledge and representations inside LLMs, one of 13 directions in Mechanistic interpretability. Builds on 52 of its 64 references that are in the map. First posted on arXiv 2 Oct 2026.

Builds on

52 of 64 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025How do Large Language Models Understand Relevance? A Mechanistic Interpretability Perspective10 Apr

Other references · 51

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  3. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  4. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  5. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  6. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  7. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  8. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  9. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
  10. 2023Steering Language Models With Activation Engineeringcited by 922
  11. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  12. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
38 more
  1. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  2. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
  3. 2019Designing and Interpreting Probes with Control Taskscited by 640
  4. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 543
  5. 2025MedGemma Technical Reportcited by 436
  6. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  7. 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 429
  8. 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 311
  9. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 300
  10. 2023LEACE: Perfect linear concept erasure in closed formcited by 267
  11. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 244
  12. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 243
  13. 2018Explainable Prediction of Medical Codes from Clinical Textcited by 234
  14. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 218
  15. 2026Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnetcited by 191
  16. 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 160
  17. 2016GRAM: Graph-based Attention Model for Healthcare Representation Learningcited by 156
  18. 2023In-Context Impersonation Reveals Large Language Models' Strengths and Biasescited by 155
  19. 2024The Remarkable Robustness of LLMs: Stages of Inference?cited by 146
  20. 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 143
  21. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 126
  22. 2025SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretabilitycited by 124
  23. 2025Are Sparse Autoencoders Useful? A Case Study in Sparse Probingcited by 115
  24. 2024Automatically Interpreting Millions of Features in Large Language Modelscited by 113
  25. 2025Sparse Autoencoders Trained on the Same Data Learn Different Featurescited by 93
  26. 2023Explaining black box text modules in natural language with language modelscited by 62
  27. 2023When "A Helpful Assistant" Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Modelscited by 53
  28. 2024Decomposing The Dark Matter of Sparse Autoencoderscited by 45
  29. 2024EHRNoteQA: An LLM Benchmark for Real-World Clinical Practice Using Discharge Summariescited by 35
  30. 2025Sparse Autoencoders for Hypothesis Generationcited by 26
  31. 2024An X-Ray Is Worth 15 Features: Sparse Autoencoders for Interpretable Radiology Report Generationcited by 25
  32. 2025Sparse Autoencoder Features for Classifications and Transferabilitycited by 21
  33. 2026Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Modelscited by 21
  34. 2024Evaluating Sparse Autoencoders on Targeted Concept Erasure Taskscited by 15
  35. 2025Resa: Transparent Reasoning Models via SAEscited by 6
  36. 2026Mechanistic Interpretability of EEG Foundation Models via Sparse Autoencoderscited by 5
  37. 2025Resurrecting the Salmon: Rethinking Mechanistic Interpretability with Domain-Specific Sparse Autoencoderscited by 3
  38. 2026CLR-voyance: Reinforcing Open-Ended Reasoning for Inpatient Clinical Decision Support with Outcome-Aware Rubricscited by 3

50 most cited of 51 All 51 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026A primer on evaluation methods for large language models in healthcare
  2. 2025The Knowledge-Reasoning Dissociation: Fundamental Limitations of LLMs in Clinical Natural Language Inference
  3. 2025Why Chain of Thought Fails in Clinical Text Understanding
  4. 2026LLM-as-a-Judge in Healthcare: A Scoping Analysis of Applications, Methods, and Human Alignment
  5. 2026A Proposed Rubric for Evaluating Expressed Clinical Reasoning in Large Language Model Responses
  6. 2026The strength of clinical evidence is recoverable from language model representations but not from their stated grades
  7. 2024Bias patterns in the application of LLMs for clinical decision support: A comprehensive study
  8. 2025Reviewing Clinical Knowledge in Medical Large Language Models: Training and Beyond