A Differentiable Atari VCS:A Complex, Fully Known Ground Truth for Explainable AI
Not on the map. Why?
Abstract (arXiv)
Explanation requires ground truth: to verify an account of a system we must know its inner functioning-just what is missing where explainable AI (XAI) is most needed. Systems we can study fall into two camps. Simple, procedural one-decision trees, rule lists, sparse linear models-have a known but trivial mechanism, so explaining them tests nothing; genuinely complex ones-deep networks, real-world tasks-need XAI but have no ground-truth inner functioning, so an explanation can be plausible, confident, and wrong with no way to tell. We remove this dichotomy with a study object both genuinely complex and fully specified-inspectable by construction-and, so gradient methods apply, fully differentiable. We reimplement the Atari 2600 Video Computer System (VCS)-a real computer architecture, and the cradle of deep reinforcement learning-as two independent end-to-end differentiable emulators in Julia (jutari) and JAX (jaxtari), each validated bit-for-bit against xitari. Both reproduce xitari on all 64 supported Arcade Learning Environment (ALE) games: 64/64 byte-identical RAM and 64/64 pixel-identical screens. Treating the cartridge ROM as a weight tensor, RAM as a soft tape, and control flow as gates, we prove the differentiable (soft) execution equals the original (hard) one bit-for-bit in the forward pass at any finite temperature, while exposing surrogate gradients where the bit logic has none. The JAX port also opens a GPU path: batched differentiable rollouts reach millions of environment-steps/s on one commodity GPU. The system was built in roughly 137 active hours over 29 calendar days, much of it written autonomously by coding agents. This paper builds and validates the foundation, showing-theoretically and in a qualitative gradient study-that gradient-based XAI on it is feasible. Both ports' full code is available under the MIT license at https://github.com/akmaier/UnderstandingVCS.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 30 references that are in the map. First posted on arXiv 21 Jun 2026.
Builds on
14 of 30 references.
References · 14
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
- 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
- 2017Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networkscited by 939
- 2014Neural Turing Machinescited by 644
- 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
- 2017Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agentscited by 269
- 2017Visualizing and Understanding Atari Agentscited by 141
- 2018Don't Unroll Adjoint: Differentiating SSA-Form Programscited by 45
- 2023OCAtari: Object-Centric Atari 2600 Reinforcement Learning Environmentscited by 30
- 2018An Atari Model Zoo for Analyzing, Visualizing, and Comparing Deep Reinforcement Learning Agentscited by 15
- 2019Free-Lunch Saliency via Attention in Atari Agentscited by 12
2 more
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Towards Faithful Agentic XAI: A Verification Method and an Open-World Benchmark for Better Model Faithfulness
- 2024XRL-Bench: A Benchmark for Evaluating and Comparing Explainable Reinforcement Learning Techniques
- 2024Three Pathways to Neurosymbolic Reinforcement Learning with Interpretable Model and Policy Networks
- 2024Abstracted Trajectory Visualization for Explainability in Reinforcement Learning
- 2024Interpretable end-to-end Neurosymbolic Reinforcement Learning agents
- 2024A quantum-classical reinforcement learning model to play Atari games
- 2020Mastering Atari with Discrete World Models
- 2026Explainable AI: Learning from the Learners