mlsanityData to 14 Oct 2026

Meta-Models: An Architecture for Decoding LLM Behaviors Through Interpreted Embeddings and Natural Language

arXiv 2410.02472 ↗First posted 3 Oct 2024cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

As Large Language Models (LLMs) become increasingly integrated into our daily lives, the potential harms from deceptive behavior underlie the need for faithfully interpreting their decision-making. While traditional probing methods have shown some effectiveness, they remain best for narrowly scoped tasks while more comprehensive explanations are still necessary. To this end, we investigate meta-models-an architecture using a "meta-model" that takes activations from an "input-model" and answers natural language questions about the input-model's behaviors. We evaluate the meta-model's ability to generalize by training them on selected task types and assessing their out-of-distribution performance in deceptive scenarios. Our findings show that meta-models generalize well to out-of-distribution tasks and point towards opportunities for future research in this area. Our code is available at https://github.com/acostarelli/meta-models-public .

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 27 of its 39 references that are in the map. First posted on arXiv 3 Oct 2024.

Builds on

27 of 39 references.

References · 27

  1. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  3. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  4. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  5. 2016HyperNetworkscited by 1,027
  6. 2022Toy Models of Superpositioncited by 971
  7. 2023Steering Language Models With Activation Engineeringcited by 922
  8. 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 486
  9. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 431
  10. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  11. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 334
  12. 2023Explainability for Large Language Models: A Surveycited by 333
15 more
  1. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 319
  2. 2021Factual Probing Is [MASK]: Learning vs. Learning to Recallcited by 291
  3. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 235
  4. 2019Detecting AI Trojans Using Meta Neural Analysiscited by 199
  5. 2019Universal Litmus Patterns: Revealing Backdoor Attacks in CNNscited by 146
  6. 2020The Multilingual Amazon Reviews Corpuscited by 143
  7. 2023How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questionscited by 83
  8. 2023Still No Lie Detector for Language Models: Probing Empirical and Conceptual Roadblockscited by 78
  9. 2024SelfIE: Self-Interpretation of Large Language Model Embeddingscited by 67
  10. 2024A Multimodal Automated Interpretability Agentcited by 63
  11. 2022Probing Classifiers are Unreliable for Concept Removal and Detectioncited by 41
  12. 2023FIND: A Function Description Benchmark for Evaluating Interpretability Methodscited by 32
  13. 2022Probing Pretrained Models of Source Codecited by 26
  14. 2022AST-Probe: Recovering abstract syntax trees from hidden representations of pre-trained language modelscited by 16
  15. 2021Hyper-Representations: Self-Supervised Representation Learning on Neural Network Weights for Model Characteristic Predictioncited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 1.

5 papers.

  1. 2025Activation Oracles: Training and Evaluating LLMs as General-Purpose Activation Explainerscited by 52
  2. 2025Large Language Models Reasoning Abilities Under Non-Ideal Conditions After RL-Fine-Tuningcited by 7
  3. 2025Adversarial Activation Patching: A Framework for Detecting and Mitigating Emergent Deception in Safety-Aligned Transformerscited by 2
  4. 2025Thinking About Thinking: SAGE-nano's Inverse Reasoning for Self-Aware Language Modelscited by 1
  5. 2026On Emergent Capabilities and Model Mergingcited by 0