# Interpreting Reinforcement Learning Agents with Susceptibilities

arXiv: https://arxiv.org/abs/2605.08007 · HTML: https://mlsanity.com/paper/2605.08007/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 11 of its 32 references that are in the map.
- First posted on arXiv 8 May 2026.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 11 of 11)

- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures](https://mlsanity.com/paper/1802.01561/) (2018)
- [Understanding and Controlling a Maze-Solving Policy Network](https://mlsanity.com/paper/2310.08043/) (2023)
- [You Are What You Eat -- AI Alignment Requires Understanding How Data Shapes Structure and Generalisation](https://mlsanity.com/paper/2502.05475/) (2025)
- [Bayesian Influence Functions for Hessian-Free Data Attribution](https://mlsanity.com/paper/2509.26544/) (2025)
- [Structural Inference: Interpreting Small Language Models with Susceptibilities](https://mlsanity.com/paper/2504.18274/) (2025)
- [Patterning: The Dual of Interpretability](https://mlsanity.com/paper/2601.13548/) (2026)
- [Programs as Singularities](https://mlsanity.com/paper/2504.08075/) (2025)
- [Embryology of a Language Model](https://mlsanity.com/paper/2508.00331/) (2025)
- [Stagewise Reinforcement Learning and the Geometry of the Regret Landscape](https://mlsanity.com/paper/2601.07524/) (2026)
- [Towards Spectroscopy: Susceptibility Clusters in Language Models](https://mlsanity.com/paper/2601.12703/) (2026)

## Built on by (most cited first, 1 of 1)

- [Verifier Errors in RLVR: Reward Hacking, Limits of Feedback, and Selective Control](https://mlsanity.com/paper/2609.35677/) (2026)

## Abstract (arXiv)

Susceptibilities are a technique for neural network interpretability that studies the response of posterior expectation values of observables to perturbations of the loss. We generalize this construction to the setting of the regret in deep reinforcement learning and investigate the utility of susceptibilities in a simple gridworld model that nevertheless exhibits non-trivial stagewise development. We argue that susceptibilities reveal internal features of the development of the model in parameter space that one cannot detect purely by studying the development of the learned policy. We validate these results with activation-steering, and discuss the framework's extension to RLHF post-training.