mlsanityData to 7 Oct 2026

Representation Learning in Deep RL via Discrete Information Bottleneck

arXiv 2212.13835 ↗First posted 28 Dec 2022cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

Several self-supervised representation learning methods have been proposed for reinforcement learning (RL) with rich observations. For real-world applications of RL, recovering underlying latent states is crucial, particularly when sensory inputs contain irrelevant and exogenous information. In this work, we study how information bottlenecks can be used to construct latent states efficiently in the presence of task-irrelevant information. We propose architectures that utilize variational and discrete information bottlenecks, coined as RepDIB, to learn structured factorized representations. Exploiting the expressiveness bought by factorized representations, we introduce a simple, yet effective, bottleneck that can be integrated with any existing self-supervised objective for RL. We demonstrate this across several online and offline RL benchmarks, along with a real robot arm task, where we find that compressed representations with RepDIB can lead to strong performance improvements, as the learned bottlenecks help predict only the relevant state while ignoring irrelevant information.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 49 of its 64 references that are in the map. First posted on arXiv 28 Dec 2022.

Builds on

49 of 64 references.

References · 49

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2017Neural Discrete Representation Learningcited by 4,650
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2015Continuous control with deep reinforcement learningcited by 3,279
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  6. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  7. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  8. 2017Deeper, Broader and Artier Domain Generalizationcited by 983
  9. 2018Exploration by Random Network Distillationcited by 966
  10. 2016Deep Variational Information Bottleneckcited by 915
  11. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 899
  12. 2015Deep Learning and the Information Bottleneck Principlecited by 793
37 more
  1. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  2. 2016Unifying Count-Based Exploration and Intrinsic Motivationcited by 632
  3. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  4. 2017Opening the Black Box of Deep Neural Networks via Informationcited by 494
  5. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 400
  6. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
  7. 2020Decoupling Representation Learning from Reinforcement Learningcited by 263
  8. 2017Count-Based Exploration with Neural Density Modelscited by 255
  9. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularizationcited by 237
  10. 2021The Statistical Complexity of Interactive Decision Makingcited by 208
  11. 2021Reinforcement Learning with Prototypical Representationscited by 179
  12. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  13. 2019Provably efficient RL with Rich Observations via Latent State Decodingcited by 169
  14. 2017Deep Exploration via Randomized Value Functionscited by 155
  15. 2015Variational Information Maximisation for Intrinsically Motivated Reinforcement Learningcited by 142
  16. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
  17. 2019Kinematic State Abstraction and Provably Efficient Rich-Observation Reinforcement Learningcited by 114
  18. 2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
  19. 2021Representation Matters: Offline Pretraining for Sequential Decision Makingcited by 90
  20. 2021Graph Structure Learning with Variational Information Bottleneckcited by 81
  21. 2020Deep Reinforcement and InfoMax Learningcited by 79
  22. 2019InfoBot: Transfer and Exploration via the Information Bottleneckcited by 78
  23. 2022Rethinking Minimal Sufficient Representation in Contrastive Learningcited by 67
  24. 2022Challenges and Opportunities in Offline Reinforcement Learning from Visual Observationscited by 63
  25. 2021Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environmentcited by 41
  26. 2021Discrete-Valued Neural Communicationcited by 39
  27. 2021Provable Representation Learning for Imitation with Contrastive Fourier Featurescited by 34
  28. 2019Reinforcement Learning with Competitive Ensembles of Information-Constrained Primitivescited by 30
  29. 2020HAMLET: A Hierarchical Multimodal Attention-based Human Activity Recognition Algorithmcited by 29
  30. 2021Provable RL with Exogenous Distractors via Multistep Inverse Dynamicscited by 28
  31. 2022Sample-Efficient Reinforcement Learning in the Presence of Exogenous Informationcited by 26
  32. 2021Dynamic Bottleneck for Robust Self-Supervised Explorationcited by 21
  33. 2021Drop-Bottleneck: Learning Discrete Compressed Representation for Noise-Robust Explorationcited by 15
  34. 2020The Variational Bandwidth Bottleneck: Stochastic Evaluation on an Information Budgetcited by 13
  35. 2022Adaptive Discrete Communication Bottlenecks with Dynamic Vector Quantizationcited by 11
  36. 2021Going Beyond Linear RL: Sample Efficient Neural Function Approximationcited by 9
  37. 2019Learning Representations in Reinforcement Learning:An Information Bottleneck Approachcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 2; 2025: 6; 2026: 2.

13 papers.

Sort:

By area:

  1. 2025A Survey of State Representation Learning for Deep Reinforcement Learningcited by 17
  2. 2023Ignorance is Bliss: Robust Control via Information Gatingcited by 10
  3. 2025On Rollouts in Model-Based Reinforcement Learningcited by 10
  4. 2024DM-Codec: Distilling Multimodal Representations for Speech Tokenizationcited by 5
  5. 2025VIBE: Annotation-Free Video-to-Text Information Bottleneck Evaluation for TL;DRcited by 3
  6. 2025Geometry of Neural Reinforcement Learning in Continuous State and Action Spacescited by 3
  7. 2025Graph-Enhanced Policy Optimization in LLM Agent Trainingcited by 2
  8. 2023Human-Guided Complexity-Controlled Abstractionscited by 1
  9. 2024Cognitively Inspired Energy-Based World Modelscited by 1
  10. 2023Distilled Mid-Fusion Transformer Networks for Multi-Modal Human Activity Recognitioncited by 0
  11. 2025Task-Driven Discrete Representation Learningcited by 0
  12. 2026Distilling a Modular Reservoir Through a Genomic Bottleneckcited by 0
1 more
  1. 2026What Must a World Model Distinguish for Planning?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Provably Efficient Representation Selection in Low-rank Markov Decision Processes: From Online to Offline RL
  2. 2024Foundation Policies with Hilbert Representations
  3. 2021Representation Learning for Online and Offline RL in Low-rank MDPs
  4. 2023Learning World Models with Identifiable Factorization
  5. 2026Skill Reuse as Compression in Agentic RL
  6. 2025Boosting Action-Information via a Variational Bottleneck on Unlabelled Robot Videos
  7. 2023Rethinking Model-based, Policy-based, and Value-based Reinforcement Learning via the Lens of Representation Complexity
  8. 2026Representation Learning Enables Scalable Multitask Deep Reinforcement Learning