# Neural Transparency: Mechanistic Interpretability Interfaces for Anticipating Model Behaviors for Personalized AI arXiv: https://arxiv.org/abs/2511.00230 ยท HTML: https://mlsanity.com/paper/2511.00230/ ## Key facts - Cited by 3 papers in the map (data through 14 Oct 2026). - Placed in Probing deception, bias and beliefs, one of 13 directions in Mechanistic interpretability. - Builds on 37 of its 73 references that are in the map. - First posted on arXiv 31 Oct 2025. ## Placed in - Mechanistic interpretability / [Probing deception, bias and beliefs](https://mlsanity.com/mechinterp/probing-deception-bias-and-beliefs/) ## Builds on (most cited first, 30 of 37) - [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024) - [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023) - [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017) - [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022) - [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021) - [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023) - [Jailbreaking Black Box Large Language Models in Twenty Queries](https://mlsanity.com/paper/2310.08419/) (2023) - [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016) - [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023) - [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023) - [Refusal in Language Models Is Mediated by a Single Direction](https://mlsanity.com/paper/2406.11717/) (2024) - [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023) - [Progress measures for grokking via mechanistic interpretability](https://mlsanity.com/paper/2301.05217/) (2023) - [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023) - [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023) - [DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models](https://mlsanity.com/paper/2306.11698/) (2023) - [Mechanistic Interpretability for AI Safety -- A Review](https://mlsanity.com/paper/2404.14082/) (2024) - [Persona Vectors: Monitoring and Controlling Character Traits in Language Models](https://mlsanity.com/paper/2507.21509/) (2025) - [Emergent Linear Representations in World Models of Self-Supervised Sequence Models](https://mlsanity.com/paper/2309.00941/) (2023) - [The Dark Side of AI Companionship: A Taxonomy of Harmful Algorithmic Behaviors in Human-AI Relationships](https://mlsanity.com/paper/2410.20130/) (2024) - [CharacterChat: Learning towards Conversational AI with Personalized Social Support](https://mlsanity.com/paper/2308.10278/) (2023) - [Why human-AI relationships need socioaffective alignment](https://mlsanity.com/paper/2502.02528/) (2025) - [Designing a Dashboard for Transparency and Control of Conversational AI](https://mlsanity.com/paper/2406.07882/) (2024) - [Investigating Affective Use and Emotional Well-being on ChatGPT](https://mlsanity.com/paper/2504.03888/) (2025) - [LLM Agents in Interaction: Measuring Personality Consistency and Linguistic Alignment in Interacting Populations of Large Language Models](https://mlsanity.com/paper/2402.02896/) (2024) - [Persona is a Double-edged Sword: Mitigating the Negative Impact of Role-playing Prompts in Zero-shot Reasoning Tasks](https://mlsanity.com/paper/2408.08631/) (2024) - [Persistent Instability in LLM's Personality Measurements: Effects of Scale, Reasoning, and Conversation History](https://mlsanity.com/paper/2508.04826/) (2025) - [Examining Identity Drift in Conversations of LLM Agents](https://mlsanity.com/paper/2412.00804/) (2024) - [Training language models to be warm and empathetic makes them less reliable and more sycophantic](https://mlsanity.com/paper/2507.21919/) (2025) - ["My Boyfriend is AI": A Computational Analysis of Human-AI Companionship in Reddit's AI Community](https://mlsanity.com/paper/2509.11391/) (2025) ## Built on by (most cited first, 3 of 3) - [Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models](https://mlsanity.com/paper/2601.14004/) (2026) - [Verbalizing LLMs' assumptions to explain and control sycophancy](https://mlsanity.com/paper/2604.03058/) (2026) - [Reflections and New Directions for Human-Centered Large Language Models](https://mlsanity.com/paper/2605.06901/) (2026) ## Abstract (arXiv) Millions of users now design personalized LLM-based chatbots that shape their daily interactions, yet they can only roughly anticipate how their design choices will manifest as behaviors in deployment. This opacity is consequential: seemingly innocuous prompts can trigger excessive sycophancy, toxicity, or other undesirable traits, degrading utility and raising safety concerns. To address this issue, we introduce an interface that enables neural transparency by exposing language model internals during chatbot design. Our approach extracts behavioral trait vectors (empathy, toxicity, sycophancy, etc.) by computing differences in neural activations between contrastive system prompts that elicit opposing behaviors. We predict chatbot behaviors by projecting the system prompt's final token activations onto these trait vectors, normalizing for cross-trait comparability, and visualizing results via an interactive sunburst diagram. To evaluate this approach, we conducted an online user study using Prolific to compare our neural transparency interface against a baseline chatbot interface without any form of transparency. Our analyses suggest that users systematically miscalibrated AI behavior: participants misjudged trait activations for eleven of fifteen analyzable traits, motivating the need for transparency tools in everyday human-AI interaction. While our interface did not change design iteration patterns, it significantly increased user trust and was enthusiastically received. Qualitative analysis revealed nuanced user experiences with the visualization, suggesting interface and interaction improvements for future work. This work offers a path for how mechanistic interpretability can be operationalized for non-technical users, establishing a foundation for safer, more aligned human-AI interactions.