# Validation-Gated Causal Interventions for Interpreting High-Stakes Large Language Model Behavior: A Case Study in Suicidality Detection

arXiv: https://arxiv.org/abs/2606.21078 · HTML: https://mlsanity.com/paper/2606.21078/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 45 references that are in the map.
- First posted on arXiv 19 Jun 2026.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [Steering Llama 2 via Contrastive Activation Addition](https://mlsanity.com/paper/2312.06681/) (2023)
- [Refusal in Language Models Is Mediated by a Single Direction](https://mlsanity.com/paper/2406.11717/) (2024)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [Eliciting Latent Predictions from Transformers with the Tuned Lens](https://mlsanity.com/paper/2303.08112/) (2023)
- [Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models](https://mlsanity.com/paper/2403.19647/) (2024)
- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [Analysis Methods in Neural Language Processing: A Survey](https://mlsanity.com/paper/1812.08951/) (2018)
- [Can Large Language Models Transform Computational Social Science?](https://mlsanity.com/paper/2305.03514/) (2023)
- [How to use and interpret activation patching](https://mlsanity.com/paper/2404.15255/) (2024)
- [Attribution Patching Outperforms Automated Circuit Discovery](https://mlsanity.com/paper/2310.10348/) (2023)
- [MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare](https://mlsanity.com/paper/2110.15621/) (2021)
- [Analyzing the Generalization and Reliability of Steering Vectors](https://mlsanity.com/paper/2407.12404/) (2024)
- [AtP*: An efficient and scalable method for localizing LLM behaviour to components](https://mlsanity.com/paper/2403.00745/) (2024)
- [Deep Learning for Suicide and Depression Identification with Unsupervised Label Correction](https://mlsanity.com/paper/2102.09427/) (2021)
- [Toward universal steering and monitoring of AI models](https://mlsanity.com/paper/2502.03708/) (2025)
- [Interpretability without actionability: mechanistic methods cannot correct language model errors despite near-perfect internal representations](https://mlsanity.com/paper/2603.18353/) (2026)
- [Monotonic Representation of Numeric Properties in Language Models](https://mlsanity.com/paper/2403.10381/) (2024)
- [Can Large Language Models Identify Implicit Suicidal Ideation? An Empirical Evaluation](https://mlsanity.com/paper/2502.17899/) (2025)
- [A Comparative Analysis of Transformer and LSTM Models for Detecting Suicidal Ideation on Reddit](https://mlsanity.com/paper/2411.15404/) (2024)
- [How do Large Language Models Understand Relevance? A Mechanistic Interpretability Perspective](https://mlsanity.com/paper/2504.07898/) (2025)
- [Beyond Behavioural Trade-Offs: Mechanistic Tracing of Pain-Pleasure Decisions in an LLM](https://mlsanity.com/paper/2602.19159/) (2026)
- [Beyond Accuracy: Interpreting Topic Representation in Suicide Ideation Detection Models](https://mlsanity.com/paper/2606.07714/) (2026)

## Abstract (arXiv)

Large language models are proposed to flag suicidal content, but representing a concept and acting on it are distinct. A suicidality feature is decodable from 0.5B parameters upward, while the behavior that uses it emerges only in the low billions. Where it emerges, it rests on a compact mid-network feature: in Llama-3.1-8B-Instruct this direction decodes suicidal from non-suicidal text above a matched lexical baseline and survives removal of the 38 most label-predictive keywords. Projecting it out drops the model's ranking from AUC 0.975 to 0.832, while an equal-norm random direction does not; ablating a low-dimensional subspace degrades it further, though no stable rank is recoverable. The direction transfers to two further corpora, replicates in Qwen and Mistral, and tracks clinician-assigned risk. Steering raises an off-target readout as much as the target across five concepts, so sufficiency claims from yes/no readouts are confounded by generic salience.