mlsanityData to 7 Oct 2026

Understanding and Controlling a Maze-Solving Policy Network

arXiv 2310.08043 ↗First posted 12 Oct 2023cs.AIcited by 19

Not on the map. Why?

Abstract (arXiv)

To understand the goals and goal representations of AI systems, we carefully study a pretrained reinforcement learning policy that solves mazes by navigating to a range of target squares. We find this network pursues multiple context-dependent goals, and we further identify circuits within the network that correspond to one of these goals. In particular, we identified eleven channels that track the location of the goal. By modifying these channels, either with hand-designed interventions or by combining forward passes, we can partially control the policy. We show that this network contains redundant, distributed, and retargetable goal representations, shedding light on the nature of goal-direction in trained policy networks.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 25 of its 40 references that are in the map. First posted on arXiv 12 Oct 2023.

Builds on

25 of 40 references.

References · 25

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2017Deep reinforcement learning from human preferencescited by 3,846
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  7. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  8. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  9. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  10. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  11. 2023Steering Language Models With Activation Engineeringcited by 824
  12. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
13 more
  1. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  2. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  3. 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
  4. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  5. 2018A Benchmark for Interpretability Methods in Deep Neural Networkscited by 467
  6. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  7. 2020A Survey on Neural Network Interpretabilitycited by 291
  8. 2022The Alignment Problem from a Deep Learning Perspectivecited by 246
  9. 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 217
  10. 2023Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchillacited by 145
  11. 2021Goal Misgeneralization in Deep Reinforcement Learningcited by 122
  12. 2022Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goalscited by 120
  13. 2020On Interpretability of Artificial Neural Networks: A Surveycited by 88

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 9; 2025: 6; 2026: 2.

19 papers.

Sort:

By area:

  1. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  2. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 240
  3. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
  4. 2024A Mechanistic Analysis of a Transformer Trained on a Symbolic Multi-Step Reasoning Taskcited by 49
  5. 2024Controlling Large Language Model Agents with Entropic Activation Steeringcited by 29
  6. 2024Quantifying Feature Space Universality Across Large Language Models via Sparse Autoencoderscited by 25
  7. 2024Planning in a recurrent neural network that plays Sokobancited by 17
  8. 2025Improved Representation Steering for Language Modelscited by 17
  9. 2023Structured World Representations in Maze-Solving Transformerscited by 16
  10. 2025Probing the Representational Power of Sparse Autoencoders in Vision Modelscited by 9
  11. 2024On Logical Extrapolation for Mazes with Recurrent and Implicit Networkscited by 7
  12. 2024Interpretability in Action: Exploratory Analysis of VPT, a Minecraft Agentcited by 4
7 more
  1. 2025Partner Modelling Emerges in Recurrent Agents (But Only When It Matters)cited by 4
  2. 2025Path Channels and Plan Extension Kernels: a Mechanistic Description of Planning in a Sokoban RNNcited by 4
  3. 2026Stagewise Reinforcement Learning and the Geometry of the Regret Landscapecited by 3
  4. 2024Mechanistic Interpretability of Reinforcement Learning Agentscited by 0
  5. 2024Debias your Large Multi-Modal Model at Test-Time via Non-Contrastive Visual Attribute Steeringcited by 0
  6. 2025Efficient Knowledge Feeding to Language Models: A Novel Integrated Encoder-Decoder Architecturecited by 0
  7. 2026Understanding Goal Generalisation in Sequential Reinforcement Learningcited by 0