# Meta-Models: An Architecture for Decoding LLM Behaviors Through Interpreted Embeddings and Natural Language

arXiv: https://arxiv.org/abs/2410.02472 · HTML: https://mlsanity.com/paper/2410.02472/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 39 references that are in the map.
- First posted on arXiv 3 Oct 2024.

## Builds on (most cited first, 27 of 27)

- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [GoEmotions: A Dataset of Fine-Grained Emotions](https://mlsanity.com/paper/2005.00547/) (2020)
- [Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task](https://mlsanity.com/paper/2210.13382/) (2022)
- [Mechanistic Interpretability for AI Safety -- A Review](https://mlsanity.com/paper/2404.14082/) (2024)
- [Finding Neurons in a Haystack: Case Studies with Sparse Probing](https://mlsanity.com/paper/2305.01610/) (2023)
- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [Emergent Linear Representations in World Models of Self-Supervised Sequence Models](https://mlsanity.com/paper/2309.00941/) (2023)
- [Factual Probing Is [MASK]: Learning vs. Learning to Recall](https://mlsanity.com/paper/2104.05240/) (2021)
- [Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models](https://mlsanity.com/paper/2401.06102/) (2024)
- [Detecting AI Trojans Using Meta Neural Analysis](https://mlsanity.com/paper/1910.03137/) (2019)
- [Universal Litmus Patterns: Revealing Backdoor Attacks in CNNs](https://mlsanity.com/paper/1906.10842/) (2019)
- [The Multilingual Amazon Reviews Corpus](https://mlsanity.com/paper/2010.02573/) (2020)
- [How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions](https://mlsanity.com/paper/2309.15840/) (2023)
- [Still No Lie Detector for Language Models: Probing Empirical and Conceptual Roadblocks](https://mlsanity.com/paper/2307.00175/) (2023)
- [SelfIE: Self-Interpretation of Large Language Model Embeddings](https://mlsanity.com/paper/2403.10949/) (2024)
- [A Multimodal Automated Interpretability Agent](https://mlsanity.com/paper/2404.14394/) (2024)
- [Probing Classifiers are Unreliable for Concept Removal and Detection](https://mlsanity.com/paper/2207.04153/) (2022)
- [FIND: A Function Description Benchmark for Evaluating Interpretability Methods](https://mlsanity.com/paper/2309.03886/) (2023)
- [Probing Pretrained Models of Source Code](https://mlsanity.com/paper/2202.08975/) (2022)
- [AST-Probe: Recovering abstract syntax trees from hidden representations of pre-trained language models](https://mlsanity.com/paper/2206.11719/) (2022)
- [Hyper-Representations: Self-Supervised Representation Learning on Neural Network Weights for Model Characteristic Prediction](https://mlsanity.com/paper/2110.15288/) (2021)

## Built on by (most cited first, 5 of 5)

- [Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainers](https://mlsanity.com/paper/2512.15674/) (2025)
- [Large Language Models Reasoning Abilities Under Non-Ideal Conditions After RL-Fine-Tuning](https://mlsanity.com/paper/2508.04848/) (2025)
- [Adversarial Activation Patching: A Framework for Detecting and Mitigating Emergent Deception in Safety-Aligned Transformers](https://mlsanity.com/paper/2507.09406/) (2025)
- [Thinking About Thinking: SAGE-nano's Inverse Reasoning for Self-Aware Language Models](https://mlsanity.com/paper/2507.00092/) (2025)
- [On Emergent Capabilities and Model Merging](https://mlsanity.com/paper/2609.24504/) (2026)

## Abstract (arXiv)

As Large Language Models (LLMs) become increasingly integrated into our daily lives, the potential harms from deceptive behavior underlie the need for faithfully interpreting their decision-making. While traditional probing methods have shown some effectiveness, they remain best for narrowly scoped tasks while more comprehensive explanations are still necessary. To this end, we investigate meta-models-an architecture using a "meta-model" that takes activations from an "input-model" and answers natural language questions about the input-model's behaviors. We evaluate the meta-model's ability to generalize by training them on selected task types and assessing their out-of-distribution performance in deceptive scenarios. Our findings show that meta-models generalize well to out-of-distribution tasks and point towards opportunities for future research in this area. Our code is available at https://github.com/acostarelli/meta-models-public .