# KnowThyself: An Agentic Assistant for LLM Interpretability

arXiv: https://arxiv.org/abs/2511.03878 · HTML: https://mlsanity.com/paper/2511.03878/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 13 of its 21 references that are in the map.
- First posted on arXiv 5 Nov 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://mlsanity.com/paper/2009.11462/) (2020)
- [A Comprehensive Overview of Large Language Models](https://mlsanity.com/paper/2307.06435/) (2023)
- [The Woman Worked as a Babysitter: On Biases in Language Generation](https://mlsanity.com/paper/1909.01326/) (2019)
- [BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation](https://mlsanity.com/paper/2101.11718/) (2021)
- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [A Multiscale Visualization of Attention in the Transformer Model](https://mlsanity.com/paper/1906.05714/) (2019)
- [Nomic Embed: Training a Reproducible Long Context Text Embedder](https://mlsanity.com/paper/2402.01613/) (2024)
- [Transcoders Find Interpretable LLM Feature Circuits](https://mlsanity.com/paper/2406.11944/) (2024)
- [Rethinking Interpretability in the Era of Large Language Models](https://mlsanity.com/paper/2402.01761/) (2024)
- [A Survey of Large Language Models Attribution](https://mlsanity.com/paper/2311.03731/) (2023)
- [Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Era](https://mlsanity.com/paper/2403.08946/) (2024)
- [LLM Attributor: Interactive Visual Attribution for LLM Generation](https://mlsanity.com/paper/2404.01361/) (2024)

## Built on by (most cited first, 5 of 5)

- [SAGE: An Agentic Explainer Framework for Interpreting SAE Features in Language Models](https://mlsanity.com/paper/2511.20820/) (2025)
- [NeuronScope: A Multi-Agent Framework for Explaining Polysemantic Neurons in Language Models](https://mlsanity.com/paper/2601.03671/) (2026)
- [SHM-Agents: A Generalist-Specialist Integrated Agent System for Structural Health Monitoring](https://mlsanity.com/paper/2605.12916/) (2026)
- [Towards Self-Evolving Agents: A Human-Inspired Adaptive Exploration-Exploitation Framework for Genetic Network Programming](https://mlsanity.com/paper/2607.11913/) (2026)
- [What Process Evaluation of Coding Agents Actually Measures: Action, Task, and Step Are Three Different Levels](https://mlsanity.com/paper/2608.22960/) (2026)

## Abstract (arXiv)

We develop KnowThyself, an agentic assistant that advances large language model (LLM) interpretability. Existing tools provide useful insights but remain fragmented and code-intensive. KnowThyself consolidates these capabilities into a chat-based interface, where users can upload models, pose natural language questions, and obtain interactive visualizations with guided explanations. At its core, an orchestrator LLM first reformulates user queries, an agent router further directs them to specialized modules, and the outputs are finally contextualized into coherent explanations. This design lowers technical barriers and provides an extensible platform for LLM inspection. By embedding the whole process into a conversational workflow, KnowThyself offers a robust foundation for accessible LLM interpretability.