mlsanityData to 7 Oct 2026

Explaining Deep Reinforcement Learning Agents In The Atari Domain through a Surrogate Model

arXiv 2110.03184 ↗First posted 7 Oct 2021cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

One major barrier to applications of deep Reinforcement Learning (RL) both inside and outside of games is the lack of explainability. In this paper, we describe a lightweight and effective method to derive explanations for deep RL agents, which we evaluate in the Atari domain. Our method relies on a transformation of the pixel-based input of the RL agent to an interpretable, percept-like input representation. We then train a surrogate model, which is itself interpretable, to replicate the behavior of the target, deep RL agent. Our experiments demonstrate that we can learn an effective surrogate that accurately approximates the underlying decision making of a target agent on a suite of Atari games.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 19 of its 39 references that are in the map. First posted on arXiv 7 Oct 2021.

Builds on

19 of 39 references.

References · 19

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  4. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  5. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  6. 2015Prioritized Experience Replaycited by 1,070
  7. 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
  8. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  9. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  10. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 301
  11. 2018Consistent Individualized Feature Attribution for Tree Ensemblescited by 279
  12. 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 267
7 more
  1. 2016Model-Agnostic Interpretability of Machine Learningcited by 219
  2. 2019Explainable Reinforcement Learning Through a Causal Lenscited by 177
  3. 2017Visualizing and Understanding Atari Agentscited by 141
  4. 2017What do we need to build explainable AI systems for the medical domain?cited by 127
  5. 2019Automated Rationale Generation: A Technique for Explainable AI and its Effects on Human Perceptionscited by 81
  6. 2017Rationalization: A Neural Machine Translation Approach to Generating Natural Language Explanationscited by 37
  7. 2016An unexpected unity among methods for interpreting model predictionscited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 0; 2024: 1; 2025: 2; 2026: 3.

9 papers.

  1. 2022Explainability in Deep Reinforcement Learning, a Review into Current Methods and Applicationscited by 6
  2. 2026Formally Verifying and Explaining Sepsis Treatment Policies with COOL-MCcited by 3
  3. 2025Co-Activation Graph Analysis of Safety-Verified and Explainable Deep Reinforcement Learning Policiescited by 2
  4. 2024Enhancing Two-Player Performance Through Single-Player Knowledge Transfer: An Empirical Study on Atari 2600 Gamescited by 1
  5. 2025Building surrogate models using trajectories of agents trained by Reinforcement Learningcited by 1
  6. 2022Exploring Adaptive MCTS with TD Learning in miniXCOMcited by 0
  7. 2022Symbolic Distillation for Learned TCP Congestion Controlcited by 0
  8. 2026COOL-MC: Verifying and Explaining RL Policies for Platelet Inventory Managementcited by 0
  9. 2026COOL-MC: Verifying and Explaining RL Policies for Multi-bridge Network Maintenancecited by 0