# Compress and Control

arXiv: https://arxiv.org/abs/1411.5326 · HTML: https://mlsanity.com/paper/1411.5326/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2014.

## Built on by (most cited first, 7 of 7)

- [Language Modeling Is Compression](https://mlsanity.com/paper/2309.10668/) (2023)
- [Shaking the foundations: delusions in sequence models for interaction and control](https://mlsanity.com/paper/2110.10819/) (2021)
- [Atari-5: Distilling the Arcade Learning Environment down to Five Games](https://mlsanity.com/paper/2210.02019/) (2022)
- [Robust Predictable Control](https://mlsanity.com/paper/2109.03214/) (2021)
- [In-context learning and Occam's razor](https://mlsanity.com/paper/2410.14086/) (2024)
- [A Model-Free Universal AI](https://mlsanity.com/paper/2602.23242/) (2026)
- [Reinforcement Learning with Information-Theoretic Actuation](https://mlsanity.com/paper/2109.15147/) (2021)

## Abstract (arXiv)

This paper describes a new information-theoretic policy evaluation technique for reinforcement learning. This technique converts any compression or density model into a corresponding estimate of value. Under appropriate stationarity and ergodicity conditions, we show that the use of a sufficiently powerful model gives rise to a consistent value function estimator. We also study the behavior of this technique when applied to various Atari 2600 video games, where the use of suboptimal modeling techniques is unavoidable. We consider three fundamentally different models, all too limited to perfectly model the dynamics of the system. Remarkably, we find that our technique provides sufficiently accurate value estimates for effective on-policy control. We conclude with a suggestive study highlighting the potential of our technique to scale to large problems.