Clinical Concept Centers in LLMs
On the map: Mechanistic interpretability → Knowledge and representations inside LLMs
Abstract (arXiv)
Large language models are increasingly used in clinical settings. However, research into the reliability and performance of these models has focused almost entirely on the language substrate, scoring what the model says. Mechanistic interpretability has found that the latent space carries a higher fidelity of representation than the text: internal representations not only encode substantially more than the output verbalizes, but the stated reasoning also systematically omits features that causally drive the answer. An evaluation of model behavior in terms of mechanistic interpretability has not been explored in clinical decision support. In this work, we extend behavioral evaluation into the latent space and ask whether clinical concepts exist as locatable, causally used representations inside open-weight LLMs. We find dedicated clinical concept centers in the latent space of all eleven open models we test. These concept centers are interpretable, firing only on their aligned clinical narratives, and meaningfully and causally drive model behavior in both constrained and open-ended settings. They are not just analytical representations, but circuits that can be utilized in clinical practice, and we explore their use from the perspective of both evaluation and performance. From the evaluation standpoint, models stay internally coherent and keep using the relevant concept centers even under adversarial role-based priming, while aligned priming improves downstream clinical performance. From a performance perspective, we simulate realistic deployment settings and find that steering models along these centers leads to meaningful downstream improvements. Finally, we conduct a blinded clinician validation and find the activation and usage of these concept centers predicts clinicians preferences.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Knowledge and representations inside LLMs, one of 13 directions in Mechanistic interpretability. Builds on 52 of its 64 references that are in the map. First posted on arXiv 2 Oct 2026.
Builds on
52 of 64 references.
Closest ancestors · 1
Same direction, newest first
- 2025How do Large Language Models Understand Relevance? A Mechanistic Interpretability Perspective10 Apr
Other references · 51
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2022Locating and Editing Factual Associations in GPTcited by 2,691
- 2022Large Language Models Encode Clinical Knowledgecited by 1,830
- 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
- 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
- 2016Understanding intermediate layers using linear classifier probescited by 1,353
- 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
- 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
- 2023Steering Language Models With Activation Engineeringcited by 922
- 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
- 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
38 more
- 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
- 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
- 2019Designing and Interpreting Probes with Control Taskscited by 640
- 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 543
- 2025MedGemma Technical Reportcited by 436
- 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
- 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 429
- 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 311
- 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 300
- 2023LEACE: Perfect linear concept erasure in closed formcited by 267
- 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 244
- 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 243
- 2018Explainable Prediction of Medical Codes from Clinical Textcited by 234
- 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 218
- 2026Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnetcited by 191
- 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 160
- 2016GRAM: Graph-based Attention Model for Healthcare Representation Learningcited by 156
- 2023In-Context Impersonation Reveals Large Language Models' Strengths and Biasescited by 155
- 2024The Remarkable Robustness of LLMs: Stages of Inference?cited by 146
- 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 143
- 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 126
- 2025SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretabilitycited by 124
- 2025Are Sparse Autoencoders Useful? A Case Study in Sparse Probingcited by 115
- 2024Automatically Interpreting Millions of Features in Large Language Modelscited by 113
- 2025Sparse Autoencoders Trained on the Same Data Learn Different Featurescited by 93
- 2023Explaining black box text modules in natural language with language modelscited by 62
- 2023When "A Helpful Assistant" Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Modelscited by 53
- 2024Decomposing The Dark Matter of Sparse Autoencoderscited by 45
- 2024EHRNoteQA: An LLM Benchmark for Real-World Clinical Practice Using Discharge Summariescited by 35
- 2025Sparse Autoencoders for Hypothesis Generationcited by 26
- 2024An X-Ray Is Worth 15 Features: Sparse Autoencoders for Interpretable Radiology Report Generationcited by 25
- 2025Sparse Autoencoder Features for Classifications and Transferabilitycited by 21
- 2026Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Modelscited by 21
- 2024Evaluating Sparse Autoencoders on Targeted Concept Erasure Taskscited by 15
- 2025Resa: Transparent Reasoning Models via SAEscited by 6
- 2026Mechanistic Interpretability of EEG Foundation Models via Sparse Autoencoderscited by 5
- 2025Resurrecting the Salmon: Rethinking Mechanistic Interpretability with Domain-Specific Sparse Autoencoderscited by 3
- 2026CLR-voyance: Reinforcing Open-Ended Reasoning for Inpatient Clinical Decision Support with Outcome-Aware Rubricscited by 3
50 most cited of 51 All 51 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026A primer on evaluation methods for large language models in healthcare
- 2025The Knowledge-Reasoning Dissociation: Fundamental Limitations of LLMs in Clinical Natural Language Inference
- 2025Why Chain of Thought Fails in Clinical Text Understanding
- 2026LLM-as-a-Judge in Healthcare: A Scoping Analysis of Applications, Methods, and Human Alignment
- 2026A Proposed Rubric for Evaluating Expressed Clinical Reasoning in Large Language Model Responses
- 2026The strength of clinical evidence is recoverable from language model representations but not from their stated grades
- 2024Bias patterns in the application of LLMs for clinical decision support: A comprehensive study
- 2025Reviewing Clinical Knowledge in Medical Large Language Models: Training and Beyond