# Calculating Mutual Information between a Reward Maximizer and its Environment

arXiv: https://arxiv.org/abs/2602.12963 · HTML: https://mlsanity.com/paper/2602.12963/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 19 references that are in the map.
- First posted on arXiv 13 Feb 2026.

## Builds on (most cited first, 5 of 5)

- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [Understanding World or Predicting Future? A Comprehensive Survey of World Models](https://mlsanity.com/paper/2411.14499/) (2024)
- [Robust agents learn causal world models](https://mlsanity.com/paper/2402.10877/) (2024)
- [Settling the Reward Hypothesis](https://mlsanity.com/paper/2212.10420/) (2022)
- [General Agents Contain World Models, even under Partial Observability and Stochasticity](https://mlsanity.com/paper/2602.03146/) (2026)

## Built on by (most cited first, 1 of 1)

- [What Capable Agents Must Know: Selection Theorems for Robust Decision-Making under Uncertainty](https://mlsanity.com/paper/2603.02491/) (2026)

## Abstract (arXiv)

An important question in the field of AI is the extent to which successful behaviour requires an internal representation of the world. In this work, we quantify the amount of information an optimal policy provides about the underlying environment. We consider a Controlled Markov Process (CMP) with n states and m actions, assuming a uniform prior over the space of possible transition dynamics. We prove that observing a deterministic policy that is optimal for any non-constant reward function then conveys exactly n log m bits of information about the environment. Specifically, we show that the mutual information between the environment and the optimal policy is n log m bits. This bound holds across a broad class of objectives, including finite-horizon, infinite-horizon discounted, and time-averaged reward maximization. These findings provide a precise information-theoretic lower bound on the ``implicit world model'' necessary for optimality.