mlsanityData to 7 Oct 2026

A Multimodal Automated Interpretability Agent

arXiv 2404.14394 ↗First posted 22 Apr 2024cs.AI cs.CL cs.CVcited by 62

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

This paper describes MAIA, a Multimodal Automated Interpretability Agent. MAIA is a system that uses neural models to automate neural model understanding tasks like feature interpretation and failure mode discovery. It equips a pre-trained vision-language model with a set of tools that support iterative experimentation on subcomponents of other models to explain their behavior. These include tools commonly used by human interpretability researchers: for synthesizing and editing inputs, computing maximally activating exemplars from real-world datasets, and summarizing and describing experimental results. Interpretability experiments proposed by MAIA compose these tools to describe and explain system behavior. We evaluate applications of MAIA to computer vision models. We first characterize MAIA's ability to describe (neuron-level) features in learned representations of images. Across several trained models and a novel dataset of synthetic vision neurons with paired ground-truth descriptions, MAIA produces descriptions comparable to those generated by expert human experimenters. We then show that MAIA can aid in two additional interpretability tasks: reducing sensitivity to spurious features, and automatically identifying inputs likely to be mis-classified.

Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 41 of its 66 references that are in the map. First posted on arXiv 22 Apr 2024.

Builds on

41 of 66 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2023Segment Anythingcited by 8,158
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  8. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  9. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  10. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  11. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  12. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
29 more
  1. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  2. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
  3. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  4. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  5. 2018Recognition in Terra Incognitacited by 655
  6. 2024GPT-4V(ision) is a Generalist Web Agent, if Groundedcited by 547
  7. 2022Visual Programming: Compositional visual reasoning without trainingcited by 507
  8. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 428
  9. 2014Understanding Deep Image Representations by Inverting Themcited by 399
  10. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 379
  11. 2020Noise or Signal: The Role of Image Backgrounds in Object Recognitioncited by 342
  12. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  13. 2022Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classificationcited by 310
  14. 2020Understanding the Role of Individual Units in a Deep Neural Networkcited by 254
  15. 2020Compositional Explanations of Neuronscited by 172
  16. 2015Visualizing and Understanding Recurrent Networkscited by 158
  17. 2022CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networkscited by 144
  18. 2022Natural Language Descriptions of Deep Visual Featurescited by 136
  19. 2018Net2Vec: Quantifying and Explaining how Concepts are Encoded by Filters in Deep Neural Networkscited by 128
  20. 2018What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Modelscited by 124
  21. 2023Explaining black box text modules in natural language with language modelscited by 61
  22. 2020Understanding Failures of Deep Networks via Robust Feature Extractioncited by 56
  23. 2018Manifold: A Model-Agnostic Framework for Interpretation and Diagnosis of Machine Learning Modelscited by 50
  24. 2018Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failurecited by 49
  25. 2023Rigorously Assessing Natural Language Explanations of Neuronscited by 44
  26. 2020Debiasing Skin Lesion Datasets and Models? Not So Fastcited by 43
  27. 2023Spawrious: A Benchmark for Fine Control of Spurious Correlation Biasescited by 41
  28. 2021Toward a Visual Concept Vocabulary for GAN Latent Spacecited by 15
  29. 2022Diagnostics for Deep Neural Networks with Automated Copy/Paste Attackscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 23; 2026: 24.

62 papers.

Sort:

By area:

  1. 2024Automatically Interpreting Millions of Features in Large Language Modelscited by 106
  2. 2024Interpreting and Editing Vision-Language Representations to Mitigate Hallucinationscited by 103
  3. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  4. 2024What's in the Image? A Deep-Dive into the Vision of Vision Language Modelscited by 66
  5. 2024Image-of-Thought Prompting for Visual Reasoning Refinement in Multimodal Large Language Modelscited by 64
  6. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  7. 2026When Models Manipulate Manifolds: The Geometry of a Counting Taskcited by 43
  8. 2026Building Production-Ready Probes For Geminicited by 38
  9. 2025Interpreting the linear structure of vision-language model embedding spacescited by 32
  10. 2024SketchAgent: Language-Driven Sequential Sketch Generationcited by 30
  11. 2024InterpBench: Semi-Synthetic Transformers for Evaluating Mechanistic Interpretability Techniquescited by 24
  12. 2025Narrow Finetuning Leaves Clearly Readable Traces in Activation Differencescited by 24
38 more
  1. 2025Because we have LLMs, we Can and Should Pursue Agentic Interpretabilitycited by 14
  2. 2025Internal Causal Mechanisms Robustly Predict Language Model Out-of-Distribution Behaviorscited by 11
  3. 2025Predictive Concept Decoders: Training Scalable End-to-End Interpretability Assistantscited by 10
  4. 2026The Curse of Multiple Mediators: Hidden Interaction Effects in Activation Patchingcited by 9
  5. 2025HyperDAS: Towards Automating Mechanistic Interpretability with Hypernetworkscited by 8
  6. 2024Learning on Model Weights using Tree Expertscited by 7
  7. 2025Interpreting Large Text-to-Image Diffusion Models with Dictionary Learningcited by 6
  8. 2025Enhancing Concept Localization in CLIP-based Concept Bottleneck Modelscited by 6
  9. 2024Meta-Models: An Architecture for Decoding LLM Behaviors Through Interpreted Embeddings and Natural Languagecited by 5
  10. 2025BrainExplore: Large-Scale Discovery of Interpretable Visual Representations in the Human Braincited by 5
  11. 2025Classifier-to-Bias: Toward Unsupervised Automatic Bias Detection for Visual Classifierscited by 4
  12. 2025Debiasing CLIP: Interpreting and Correcting Bias in Attention Headscited by 4
  13. 2025Evaluating Neuron Explanations: A Unified Framework with Sanity Checkscited by 4
  14. 2025Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectivescited by 4
  15. 2025Measuring Sparse Autoencoder Feature Sensitivitycited by 4
  16. 2026Learning Concept Bottleneck Models from Mechanistic Explanationscited by 4
  17. 2025Deceptive Automated Interpretability: Language Models Coordinating to Fool Oversight Systemscited by 3
  18. 2025Faithful and Stable Neuron Explanations for Trustworthy Mechanistic Interpretabilitycited by 3
  19. 2026The Dual Mechanisms of Spatial Variable Binding in Vision-Language Modelscited by 3
  20. 2026Automated Interpretability and Feature Discovery in Language Models with Agentscited by 3
  21. 2024Contrastive Sparse Autoencoders for Interpreting Planning of Chess-Playing Agentscited by 2
  22. 2024Automatic benchmarking of large multimodal models via iterative experiment programmingcited by 2
  23. 2024GIFT: A Framework Towards Global Interpretable Faithful Textual Explanations of Vision Classifierscited by 2
  24. 2024Verbalized Representation Learning for Interpretable Few-Shot Generalizationcited by 2
  25. 2025Insights into a radiology-specialised multimodal large language model with sparse autoencoderscited by 2
  26. 2025Semantic Regexes: Auto-Interpreting LLM Features with a Structured Languagecited by 2
  27. 2025Automated Detection of Visual Attribute Reliance with a Self-Reflective Agentcited by 2
  28. 2026Pitfalls in Evaluating Interpretability Agentscited by 2
  29. 2025Uncovering Branch specialization in InceptionV1 using k sparse autoencoderscited by 1
  30. 2025Towards Open-Ended Visual Scientific Discovery with Sparse Autoencoderscited by 1
  31. 2026Language Models Can Explain Visual Features via Steeringcited by 1
  32. 2026Disentangling MLP Neuron Weights in Vocabulary Spacecited by 1
  33. 2026Agentic-imodels: Evolving agentic interpretability tools via autoresearchcited by 1
  34. 2026Discovering a Zeta Map Algorithm on Dyck Paths via Mechanistic Interpretabilitycited by 1
  35. 2026Self-CTRL: Self-Consistency Training with Reinforcement Learningcited by 1
  36. 2024TinyEmo: Scaling down Emotional Reasoning via Metric Projectioncited by 0
  37. 2025Distilled Circuits: A Mechanistic Study of Internal Restructuring in Knowledge Distillationcited by 0
  38. 2025Beyond Top Activations: Efficient and Reliable Crowdsourced Evaluation of Automated Interpretabilitycited by 0

50 most cited of 62 All 62 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024LVLM-Interpret: An Interpretability Tool for Large Vision-Language Models
  2. 2025A Survey on Interpretability in Visual Recognition
  3. 2024FastRM: An efficient and automatic explainability framework for multimodal generative models
  4. 2024VALE: A Multimodal Visual and Language Explanation Framework for Image Classifiers using eXplainable AI and Language Models
  5. 2026From Input to Output: A Flexible Agent for Dual-End Interpretation of Sparse Autoencoder Features
  6. 2026DEX-AR: A Dynamic Explainability Method for Autoregressive Vision-Language Models
  7. 2026Generative Interpretability via Scalable Neuro-Symbolic Models
  8. 2026From Weights to Concepts: Data-Free Interpretability of CLIP via Singular Vector Decomposition