# We Can't Understand AI Using our Existing Vocabulary

arXiv: https://arxiv.org/abs/2502.07586 · HTML: https://mlsanity.com/paper/2502.07586/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 74 references that are in the map.
- First posted on arXiv 11 Feb 2025.

## Builds on (most cited first, 30 of 44)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Large Language Monkeys: Scaling Inference Compute with Repeated Sampling](https://mlsanity.com/paper/2407.21787/) (2024)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [Adafactor: Adaptive Learning Rates with Sublinear Memory Cost](https://mlsanity.com/paper/1804.04235/) (2018)
- [Discovering Latent Knowledge in Language Models Without Supervision](https://mlsanity.com/paper/2212.03827/) (2022)
- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Designing and Interpreting Probes with Control Tasks](https://mlsanity.com/paper/1909.03368/) (2019)
- [Interpretation of Neural Networks is Fragile](https://mlsanity.com/paper/1710.10547/) (2017)
- [What do you learn from context? Probing for sentence structure in contextualized word representations](https://mlsanity.com/paper/1905.06316/) (2019)
- [The (Un)reliability of saliency methods](https://mlsanity.com/paper/1711.00867/) (2017)
- [CRAFT: Concept Recursive Activation FacTorization for Explainability](https://mlsanity.com/paper/2211.10154/) (2022)
- [Acquisition of Chess Knowledge in AlphaZero](https://mlsanity.com/paper/2111.09259/) (2021)

## Built on by (most cited first, 25 of 25)

- [MIB: A Mechanistic Interpretability Benchmark](https://mlsanity.com/paper/2504.13151/) (2025)
- [The Dual-Route Model of Induction](https://mlsanity.com/paper/2504.03022/) (2025)
- [From Data to Behavior: Predicting Unintended Model Behaviors Before Training](https://mlsanity.com/paper/2602.04735/) (2026)
- [A Mathematical Philosophy of Explanations in Mechanistic Interpretability -- The Strange Science Part I.i](https://mlsanity.com/paper/2505.00808/) (2025)
- [Crosscoding Through Time: Tracking Emergence & Consolidation Of Linguistic Representations Throughout LLM Pretraining](https://mlsanity.com/paper/2509.05291/) (2025)
- [Neologism Learning for Controllability and Self-Verbalization](https://mlsanity.com/paper/2510.08506/) (2025)
- [LLMs Should Express Uncertainty Explicitly](https://mlsanity.com/paper/2604.05306/) (2026)
- [Aligned Probing: Relating Toxic Behavior and Model Internals](https://mlsanity.com/paper/2503.13390/) (2025)
- [Representation Consistency for Accurate and Coherent LLM Answer Aggregation](https://mlsanity.com/paper/2506.21590/) (2025)
- [When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI Collaboration](https://mlsanity.com/paper/2506.05579/) (2025)
- [Foundations of Interpretable Models](https://mlsanity.com/paper/2508.00545/) (2025)
- [Skill Neologisms: Towards Skill-based Continual Learning](https://mlsanity.com/paper/2605.04970/) (2026)
- [Cognitive models can reveal interpretable value trade-offs in language models](https://mlsanity.com/paper/2506.20666/) (2025)
- [Interpreting Language Models Through Concept Descriptions: A Survey](https://mlsanity.com/paper/2510.01048/) (2025)
- [Indications of Belief-Guided Agency and Meta-Cognitive Monitoring in Large Language Models](https://mlsanity.com/paper/2602.02467/) (2026)
- [Prompting as Scientific Inquiry](https://mlsanity.com/paper/2507.00163/) (2025)
- [A Rigorous Behavior Assessment of CNNs Using a Data-Domain Sampling Regime](https://mlsanity.com/paper/2507.03866/) (2025)
- [Neologism Learning as a Parameter-Efficient Alternative to Fine-Tuning for Model Steering](https://mlsanity.com/paper/2512.18551/) (2025)
- [A Vision for Multisensory Intelligence: Sensing, Science, and Synergy](https://mlsanity.com/paper/2601.04563/) (2026)
- [White-Box Sensitivity Auditing with Steering Vectors](https://mlsanity.com/paper/2601.16398/) (2026)
- [From Found to Designed: Concepts as a Design Axis for Large Language Models](https://mlsanity.com/paper/2607.26825/) (2026)
- [An Investigation of the NeurIPS and ICML 2025 Position Tracks](https://mlsanity.com/paper/2608.16894/) (2026)
- [(V)LMs generalize beyond surface co-occurrence: Evidence from cross-modal number agreement](https://mlsanity.com/paper/2609.00443/) (2026)
- [From Interpretability Methods to Interpretable Models](https://mlsanity.com/paper/2609.05399/) (2026)
- [Medical AI Encodes a "Feeling of Error": Verifying Cancer Segmentation via Internal Concepts](https://mlsanity.com/paper/2609.08879/) (2026)

## Abstract (arXiv)

This position paper argues that, in order to understand AI, we cannot rely on our existing vocabulary of human words. Instead, we should strive to develop neologisms: new words that represent precise human concepts that we want to teach machines, or machine concepts that we need to learn. We start from the premise that humans and machines have differing concepts. This means interpretability can be framed as a communication problem: humans must be able to reference and control machine concepts, and communicate human concepts to machines. Creating a shared human-machine language through developing neologisms, we believe, could solve this communication problem. Successful neologisms achieve a useful amount of abstraction: not too detailed, so they're reusable in many contexts, and not too high-level, so they convey precise information. As a proof of concept, we demonstrate how a "length neologism" enables controlling LLM response length, while a "diversity neologism" allows sampling more variable responses. Taken together, we argue that we cannot understand AI using our existing vocabulary, and expanding it through neologisms creates opportunities for both controlling and understanding machines better.