# Dynamics Generalization via Information Bottleneck in Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2008.00614 · HTML: https://mlsanity.com/paper/2008.00614/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Aug 2020.

## Built on by (most cited first, 26 of 26)

- [A Survey of Zero-shot Generalisation in Deep Reinforcement Learning](https://mlsanity.com/paper/2111.09794/) (2021)
- [Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observability](https://mlsanity.com/paper/2107.06277/) (2021)
- [Efficient Knowledge Distillation from Model Checkpoints](https://mlsanity.com/paper/2210.06458/) (2022)
- [Robust Predictable Control](https://mlsanity.com/paper/2109.03214/) (2021)
- [Dynamics Generalisation in Reinforcement Learning via Adaptive Context-Aware Policies](https://mlsanity.com/paper/2310.16686/) (2023)
- [Selective Visual Representations Improve Convergence and Generalization for Embodied AI](https://mlsanity.com/paper/2311.04193/) (2023)
- [Learn Goal-Conditioned Policy with Intrinsic Motivation for Deep Reinforcement Learning](https://mlsanity.com/paper/2104.05043/) (2021)
- [Bayesian Imitation Learning for End-to-End Mobile Manipulation](https://mlsanity.com/paper/2202.07600/) (2022)
- [Reinforcement Learning with Simple Sequence Priors](https://mlsanity.com/paper/2305.17109/) (2023)
- [Bridging the Sim-to-Real Gap from the Information Bottleneck Perspective](https://mlsanity.com/paper/2305.18464/) (2023)
- [On Rollouts in Model-Based Reinforcement Learning](https://mlsanity.com/paper/2501.16918/) (2025)
- [Masked Imitation Learning: Discovering Environment-Invariant Modalities in Multimodal Demonstrations](https://mlsanity.com/paper/2209.07682/) (2022)
- [PID-Inspired Inductive Biases for Deep Reinforcement Learning in Partially Observable Control Tasks](https://mlsanity.com/paper/2307.05891/) (2023)
- [Generalization Capability for Imitation Learning](https://mlsanity.com/paper/2504.18538/) (2025)
- [Model-Advantage and Value-Aware Models for Model-Based Reinforcement Learning: Bridging the Gap in Theory and Practice](https://mlsanity.com/paper/2106.14080/) (2021)
- [AACC: Asymmetric Actor-Critic in Contextual Reinforcement Learning](https://mlsanity.com/paper/2208.02376/) (2022)
- [The Role of Diverse Replay for Generalisation in Reinforcement Learning](https://mlsanity.com/paper/2306.05727/) (2023)
- [Predictable Reinforcement Learning Dynamics through Entropy Rate Minimization](https://mlsanity.com/paper/2311.18703/) (2023)
- [Training on Irrelevant States Implies Data Augmentation: Generalization in Contextual MDPs](https://mlsanity.com/paper/2410.03565/) (2024)
- [Improving generalization to new environments and removing catastrophic forgetting in Reinforcement Learning by using an eco-system of agents](https://mlsanity.com/paper/2204.06550/) (2022)
- [Improving generalization in reinforcement learning through forked agents](https://mlsanity.com/paper/2212.06451/) (2022)
- [Explore-Go: Leveraging Exploration for Generalisation in Deep Reinforcement Learning](https://mlsanity.com/paper/2406.08069/) (2024)
- [TEA: Trajectory Encoding Augmentation for Robust and Transferable Policies in Offline Reinforcement Learning](https://mlsanity.com/paper/2411.19133/) (2024)
- [Maximum Total Correlation Reinforcement Learning](https://mlsanity.com/paper/2505.16734/) (2025)
- [Goals and the Structure of Experience](https://mlsanity.com/paper/2508.15013/) (2025)
- [Leveraging weights signals -- Predicting and improving generalizability in reinforcement learning](https://mlsanity.com/paper/2511.20234/) (2025)

## Abstract (arXiv)

Despite the significant progress of deep reinforcement learning (RL) in solving sequential decision making problems, RL agents often overfit to training environments and struggle to adapt to new, unseen environments. This prevents robust applications of RL in real world situations, where system dynamics may deviate wildly from the training settings. In this work, our primary contribution is to propose an information theoretic regularization objective and an annealing-based optimization method to achieve better generalization ability in RL agents. We demonstrate the extreme generalization benefits of our approach in different domains ranging from maze navigation to robotic tasks; for the first time, we show that agents can generalize to test parameters more than 10 standard deviations away from the training parameter distribution. This work provides a principled way to improve generalization in RL by gradually removing information that is redundant for task-solving; it opens doors for the systematic study of generalization from training to extremely different testing settings, focusing on the established connections between information theory and machine learning.