mlsanityData to 14 Oct 2026

Interpreting Reinforcement Learning Agents with Susceptibilities

arXiv 2605.08007 ↗First posted 8 May 2026cs.LGcited by 1

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Susceptibilities are a technique for neural network interpretability that studies the response of posterior expectation values of observables to perturbations of the loss. We generalize this construction to the setting of the regret in deep reinforcement learning and investigate the utility of susceptibilities in a simple gridworld model that nevertheless exhibits non-trivial stagewise development. We argue that susceptibilities reveal internal features of the development of the model in parameter space that one cannot detect purely by studying the development of the learned policy. We validate these results with activation-steering, and discuss the framework's extension to RLHF post-training.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 11 of its 32 references that are in the map. First posted on arXiv 8 May 2026.

Builds on

11 of 32 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026Stagewise Reinforcement Learning and the Geometry of the Regret Landscape12 Jan

Other references · 10

  1. 2023Steering Language Models With Activation Engineeringcited by 922
  2. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  3. 2023Understanding and Controlling a Maze-Solving Policy Networkcited by 22
  4. 2025You Are What You Eat -- AI Alignment Requires Understanding How Data Shapes Structure and Generalisationcited by 17
  5. 2025Bayesian Influence Functions for Hessian-Free Data Attributioncited by 14
  6. 2025Structural Inference: Interpreting Small Language Models with Susceptibilitiescited by 13
  7. 2026Patterning: The Dual of Interpretabilitycited by 7
  8. 2025Programs as Singularitiescited by 4
  9. 2025Embryology of a Language Modelcited by 4
  10. 2026Towards Spectroscopy: Susceptibility Clusters in Language Modelscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Verifier Errors in RLVR: Reward Hacking, Limits of Feedback, and Selective Controlcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Safe But Not Sorry: Reducing Over-Conservatism in Safety Critics via Uncertainty-Aware Modulation
  2. 2025Viability of Future Actions: Robust Safety in Reinforcement Learning via Entropy Regularization
  3. 2023Survival Instinct in Offline Reinforcement Learning
  4. 2026When Errors Can Be Beneficial: A Categorization of Imperfect Rewards for Policy Gradient
  5. 2024BET: Explaining Deep Reinforcement Learning through The Error-Prone Decisions
  6. 2024When Your AIs Deceive You: Challenges of Partial Observability in Reinforcement Learning from Human Feedback
  7. 2026What Capable Agents Must Know: Selection Theorems for Robust Decision-Making under Uncertainty
  8. 2026Learning When to Trust in Contextual Social Bandits