mlsanityData to 14 Oct 2026

Inducing, Detecting and Characterising Neural Modules: A Pipeline for Functional Interpretability in Reinforcement Learning

arXiv 2501.17077 ↗First posted 28 Jan 2025cs.LG cs.AIcited by 1

On the map: Deep reinforcement learning methods → Interpretable and logic-guided RL

Abstract (arXiv)

Interpretability is crucial for ensuring RL systems align with human values. However, it remains challenging to achieve in complex decision making domains. Existing methods frequently attempt interpretability at the level of fundamental model units, such as neurons or decision nodes: an approach which scales poorly to large models. Here, we instead propose an approach to interpretability at the level of functional modularity. We show how encouraging sparsity and locality in network weights leads to the emergence of functional modules in RL policy networks. To detect these modules, we develop an extended Louvain algorithm which uses a novel `correlation alignment' metric to overcome the limitations of standard network analysis techniques when applied to neural network architectures. Applying these methods to 2D and 3D MiniGrid environments reveals the consistent emergence of distinct navigational modules for different axes, and we further demonstrate how these functions can be validated through direct interventions on network weights prior to inference.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Interpretable and logic-guided RL, one of 15 directions in Deep reinforcement learning methods. Builds on 23 of its 45 references that are in the map. First posted on arXiv 28 Jan 2025.

Builds on

23 of 45 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Mitigating Information Loss in Tree-Based Reinforcement Learning via Direct Optimization16 Aug
  2. 2024Interpretable Concept Bottlenecks to Align Reinforcement Learning Agents11 Jan

Other references · 21

  1. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  2. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  3. 2022Toy Models of Superpositioncited by 971
  4. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  5. 2016The Mythos of Model Interpretabilitycited by 721
  6. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 327
  7. 2015Convergent Learning: Do different neural networks learn the same representations?cited by 220
  8. 2018Verifiable Reinforcement Learning via Policy Extractioncited by 177
  9. 2023Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learningcited by 111
  10. 2018Neural probabilistic motor primitives for humanoid controlcited by 97
  11. 2021A Survey on Interpretable Reinforcement Learningcited by 73
  12. 2019Neural Logic Reinforcement Learningcited by 53
9 more
  1. 2017Interpretable Policies for Reinforcement Learning by Genetic Programmingcited by 48
  2. 2023Interpretable and Explainable Logical Policies via Neurally Guided Symbolic Abstractioncited by 46
  3. 2023Seeing is Believing: Brain-Inspired Modular Training for Mechanistic Interpretabilitycited by 38
  4. 2024Gradient Routing: Masking Gradients to Localize Computation in Neural Networkscited by 34
  5. 2021Clusterability in Neural Networkscited by 24
  6. 2024NAVIX: Scaling MiniGrid Environments with JAXcited by 18
  7. 2024BlendRL: A Framework for Merging Symbolic and Neural Policy Learningcited by 14
  8. 2015Faster unfolding of communities: speeding up the Louvain algorithmcited by 7
  9. 2020Hierarchical principles of embodied reinforcement learning: A reviewcited by 5

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Evaluating Interpretable Reinforcement Learning by Distilling Policies into Programscited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2024Mechanistic Interpretability of Reinforcement Learning Agents
  2. 2024Towards a Research Community in Interpretable Reinforcement Learning: the InterpPol Workshop
  3. 2025Balancing Interpretability and Performance in Reinforcement Learning: An Adaptive Spectral Based Linear Approach
  4. 2026Principal Prototype Analysis on Manifold for Interpretable Reinforcement Learning
  5. 2025Interpretable by Design: Query-Specific Neural Modules for Explainable Reinforcement Learning
  6. 2025From Explainability to Interpretability: Interpretable Policies in Reinforcement Learning Via Model Explanation
  7. 2025A Comprehensive Survey on Self-Interpretable Neural Networks
  8. 2026What Does an LLM Learn from Reinforcement Learning? A Mechanistic Interpretability Perspective with Fixed-SAE Track