Validation-Gated Causal Interventions for Interpreting High-Stakes Large Language Model Behavior: A Case Study in Suicidality Detection
Not on the map. Why?
Abstract (arXiv)
Large language models are proposed to flag suicidal content, but representing a concept and acting on it are distinct. A suicidality feature is decodable from 0.5B parameters upward, while the behavior that uses it emerges only in the low billions. Where it emerges, it rests on a compact mid-network feature: in Llama-3.1-8B-Instruct this direction decodes suicidal from non-suicidal text above a matched lexical baseline and survives removal of the 38 most label-predictive keywords. Projecting it out drops the model's ranking from AUC 0.975 to 0.832, while an equal-norm random direction does not; ablating a low-dimensional subspace degrades it further, though no stable rank is recoverable. The direction transfers to two further corpora, replicates in Qwen and Mistral, and tracks clinician-assigned risk. Steering raises an off-target readout as much as the target across five concepts, so sufficiency claims from yes/no readouts are confounded by generic salience.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 45 references that are in the map. First posted on arXiv 19 Jun 2026.
Builds on
30 of 45 references.
References · 30
- 2017Attention Is All You Needcited by 60,361
- 2022Locating and Editing Factual Associations in GPTcited by 2,691
- 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
- 2016Understanding intermediate layers using linear classifier probescited by 1,353
- 2023A Survey on Evaluation of Large Language Modelscited by 1,242
- 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
- 2022Toy Models of Superpositioncited by 971
- 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
- 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
- 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
18 more
- 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
- 2023Explainability for Large Language Models: A Surveycited by 333
- 2018Analysis Methods in Neural Language Processing: A Surveycited by 277
- 2023Can Large Language Models Transform Computational Social Science?cited by 250
- 2024How to use and interpret activation patchingcited by 214
- 2023Attribution Patching Outperforms Automated Circuit Discoverycited by 209
- 2021MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcarecited by 161
- 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 143
- 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 111
- 2021Deep Learning for Suicide and Depression Identification with Unsupervised Label Correctioncited by 45
- 2025Toward universal steering and monitoring of AI modelscited by 30
- 2026Interpretability without actionability: mechanistic methods cannot correct language model errors despite near-perfect internal representationscited by 17
- 2024Monotonic Representation of Numeric Properties in Language Modelscited by 15
- 2025Can Large Language Models Identify Implicit Suicidal Ideation? An Empirical Evaluationcited by 13
- 2024A Comparative Analysis of Transformer and LSTM Models for Detecting Suicidal Ideation on Redditcited by 7
- 2025How do Large Language Models Understand Relevance? A Mechanistic Interpretability Perspectivecited by 7
- 2026Beyond Behavioural Trade-Offs: Mechanistic Tracing of Pain-Pleasure Decisions in an LLMcited by 1
- 2026Beyond Accuracy: Interpreting Topic Representation in Suicide Ideation Detection Modelscited by 1
Built on by
Not cited yet.