# Deep RL With Information Constrained Policies: Generalization in Continuous Control

arXiv: https://arxiv.org/abs/2010.04646 · HTML: https://mlsanity.com/paper/2010.04646/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Oct 2020.

## Built on by (most cited first, 6 of 6)

- [Position: Theory of Mind Benchmarks are Broken for Large Language Models](https://mlsanity.com/paper/2412.19726/) (2024)
- [Enabling Realtime Reinforcement Learning at Scale with Staggered Asynchronous Inference](https://mlsanity.com/paper/2412.14355/) (2024)
- [The Effectiveness of Approximate Regularized Replay for Efficient Supervised Fine-Tuning of Large Language Models](https://mlsanity.com/paper/2512.22337/) (2025)
- [On Policy Stochasticity in Mutual Information Optimal Control of Linear Systems](https://mlsanity.com/paper/2507.21543/) (2025)
- [Deep imagination is a close to optimal policy for planning in large decision trees under limited resources](https://mlsanity.com/paper/2104.06339/) (2021)
- [Mutual Information Optimal Density Control of Linear Systems and Generalized Schr"odinger Bridges with Reference Refinement](https://mlsanity.com/paper/2605.09349/) (2026)

## Abstract (arXiv)

Biological agents learn and act intelligently in spite of a highly limited capacity to process and store information. Many real-world problems involve continuous control, which represents a difficult task for artificial intelligence agents. In this paper we explore the potential learning advantages a natural constraint on information flow might confer onto artificial agents in continuous control tasks. We focus on the model-free reinforcement learning (RL) setting and formalize our approach in terms of an information-theoretic constraint on the complexity of learned policies. We show that our approach emerges in a principled fashion from the application of rate-distortion theory. We implement a novel Capacity-Limited Actor-Critic (CLAC) algorithm and situate it within a broader family of RL algorithms such as the Soft Actor Critic (SAC) and Mutual Information Reinforcement Learning (MIRL) algorithm. Our experiments using continuous control tasks show that compared to alternative approaches, CLAC offers improvements in generalization between training and modified test environments. This is achieved in the CLAC model while displaying the high sample efficiency of similar methods.