mlsanityData to 7 Oct 2026

Interpretability in Action: Exploratory Analysis of VPT, a Minecraft Agent

arXiv 2407.12161 ↗First posted 16 Jul 2024cs.AIcited by 4

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Understanding the mechanisms behind decisions taken by large foundation models in sequential decision making tasks is critical to ensuring that such systems operate transparently and safely. In this work, we perform exploratory analysis on the Video PreTraining (VPT) Minecraft playing agent, one of the largest open-source vision-based agents. We aim to illuminate its reasoning mechanisms by applying various interpretability techniques. First, we analyze the attention mechanism while the agent solves its training task - crafting a diamond pickaxe. The agent pays attention to the last four frames and several key-frames further back in its six-second memory. This is a possible mechanism for maintaining coherence in a task that takes 3-10 minutes, despite the short memory span. Secondly, we perform various interventions, which help us uncover a worrying case of goal misgeneralization: VPT mistakenly identifies a villager wearing brown clothes as a tree trunk when the villager is positioned stationary under green tree leaves, and punches it to death.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 27 of its 43 references that are in the map. First posted on arXiv 16 Jul 2024.

Builds on

27 of 43 references.

References · 27

  1. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  2. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  3. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  4. 2018Sanity Checks for Saliency Mapscited by 1,149
  5. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  6. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  7. 2022In-context Learning and Induction Headscited by 930
  8. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  9. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  10. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 338
  11. 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 210
  12. 2023How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelcited by 199
15 more
  1. 2024How to use and interpret activation patchingcited by 191
  2. 2023Interpreting CLIP's Image Representation via Text-Based Decompositioncited by 189
  3. 2022On Feature Learning in the Presence of Spurious Correlationscited by 145
  4. 2023Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchillacited by 145
  5. 2023A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operationscited by 141
  6. 2020Explainable Reinforcement Learning: A Surveycited by 139
  7. 2021Goal Misgeneralization in Deep Reinforcement Learningcited by 122
  8. 2023STEVE-1: A Generative Model for Text-to-Behavior in Minecraftcited by 95
  9. 2021The MultiBERTs: BERT Reproductions for Robustness Analysiscited by 89
  10. 2023Copy Suppression: Comprehensively Understanding an Attention Headcited by 73
  11. 2021A Survey on Interpretable Reinforcement Learningcited by 70
  12. 2023Explaining Reinforcement Learning with Shapley Valuescited by 26
  13. 2023Understanding and Controlling a Maze-Solving Policy Networkcited by 19
  14. 2023Towards Solving Fuzzy Tasks with Human Feedback: A Retrospective of the MineRL BASALT 2022 Competitioncited by 15
  15. 2021Interpretable Model-based Hierarchical Reinforcement Learning using Inductive Logic Programmingcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Prisma: An Open Source Toolkit for Mechanistic Interpretability in Vision and Videocited by 20
  2. 2025CombatVLA: An Efficient Vision-Language-Action Model for Combat Tasks in 3D Action Role-Playing Gamescited by 13
  3. 2025Deep RL Needs Deep Behavior Analysis: Exploring Implicit Planning by Model-Free Agents in Open-Ended Environmentscited by 4
  4. 2026How Foundational Skills Influence VLM-based Embodied Agents:A Native Perspectivecited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025VistaWise: Building Cost-Effective Agent with Cross-Modal Knowledge Graph for Minecraft
  2. 2026Interpreting Agentic Systems: Beyond Model Explanations to System-Level Accountability
  3. 2026WISE: A Long-Horizon Agent in Minecraft with Why-Which Reasoning
  4. 2025OpenHA: A Series of Open-Source Hierarchical Agentic Models in Minecraft
  5. 2025ROCKET-2: Steering Visuomotor Policy via Cross-View Goal Alignment
  6. 2024ADAM: An Embodied Causal Agent in Open-World Environments
  7. 2026MineNPC-Task: Task Suite for Memory-Aware Minecraft Agents
  8. 2026A Behavioural and Representational Evaluation of Goal-Directedness in Language Model Agents