mlsanityData to 7 Oct 2026

InfoBot: Transfer and Exploration via the Information Bottleneck

arXiv 1901.10902 ↗First posted 30 Jan 2019stat.ML cs.LGcited by 78

Not on the map. Why?

Abstract (arXiv)

A central challenge in reinforcement learning is discovering effective policies for tasks where rewards are sparsely distributed. We postulate that in the absence of useful reward signals, an effective exploration strategy should seek out it decision states. These states lie at critical junctions in the state space from where the agent can transition to new, potentially unexplored regions. We propose to learn about decision states from prior experience. By training a goal-conditioned policy with an information bottleneck, we can identify decision states by examining where the model actually leverages the goal state. We find that this simple mechanism effectively identifies decision states, even in partially observed settings. In effect, the model learns the sensory cues that correlate with potential subgoals. In new environments, this model can then identify novel subgoals for further exploration, guiding the agent through a sequence of potential decision states and through new regions of the state space.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). First posted on arXiv 30 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 18; 2022: 18; 2023: 12; 2024: 7; 2025: 15; 2026: 7.

78 papers.

Sort:

By area:

  1. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  2. 2021Rapid Exploration for Open-World Navigation with Latent Goal Modelscited by 118
  3. 2024InfoRM: Mitigating Reward Hacking in RLHF via Information-Theoretic Reward Modelingcited by 76
  4. 2021Neural Production Systems: Learning Rule-Governed Visual Dynamicscited by 75
  5. 2022Retrieval-Augmented Reinforcement Learningcited by 53
  6. 2022Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representationscited by 48
  7. 2022Vision-Based Manipulators Need to Also See from Their Handscited by 46
  8. 2021Improving Subgraph Recognition with Variational Graph Information Bottleneckcited by 45
  9. 2022Efficient Knowledge Distillation from Model Checkpointscited by 37
  10. 2021Robust Predictable Controlcited by 29
  11. 2021Recognizing Predictive Substructures with Subgraph Information Bottleneckcited by 28
  12. 2021PAC-Bayes Information Bottleneckcited by 28
38 more
  1. 2022Imitate and Repurpose: Learning Reusable Robot Movement Skills From Human and Animal Behaviorscited by 27
  2. 2021Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoningcited by 25
  3. 2023Selective Visual Representations Improve Convergence and Generalization for Embodied AIcited by 19
  4. 2022Understanding the Complexity Gains of Single-Task RL with a Curriculumcited by 16
  5. 2021Drop-Bottleneck: Learning Discrete Compressed Representation for Noise-Robust Explorationcited by 15
  6. 2021Unsupervised Domain Adaptation with Dynamics-Aware Rewards in Reinforcement Learningcited by 15
  7. 2022Bayesian Imitation Learning for End-to-End Mobile Manipulationcited by 13
  8. 2022Temporal Latent Bottleneck: Synthesis of Fast and Slow Processing Mechanisms in Sequence Learningcited by 13
  9. 2022Representation Learning in Deep RL via Discrete Information Bottleneckcited by 13
  10. 2023Reinforcement Learning with Simple Sequence Priorscited by 11
  11. 2023CLUE: Calibrated Latent Guidance for Offline Reinforcement Learningcited by 11
  12. 2023MyoDex: A Generalizable Prior for Dexterous Manipulationcited by 11
  13. 2021Action Priors for Large Action Spaces in Roboticscited by 10
  14. 2022MO2: Model-Based Offline Optionscited by 10
  15. 2022Discrete Factorial Representations as an Abstraction for Goal Conditioned Reinforcement Learningcited by 10
  16. 2023Learning High-Order Relationships of Brain Regionscited by 10
  17. 2024Simplifying Latent Dynamics with Softly State-Invariant World Modelscited by 10
  18. 2022Information bottleneck theory of high-dimensional regression: relevancy, efficiency and optimalitycited by 9
  19. 2025VRPO: Rethinking Value Modeling for Robust RL under Noisy Supervision in LLM Post-Trainingcited by 9
  20. 2021Measuring Progress in Deep Reinforcement Learning Sample Efficiencycited by 8
  21. 2021Towards an Understanding of Default Policies in Multitask Policy Optimizationcited by 8
  22. 2024Pragmatic Feature Preferences: Learning Reward-Relevant Preferences from Human Inputcited by 8
  23. 2021Toward Robust Long Range Policy Transfercited by 6
  24. 2021Perturbation Theory for the Information Bottleneckcited by 6
  25. 2022From Psychological Curiosity to Artificial Curiosity: Curiosity-Driven Learning in Artificial Intelligence Taskscited by 6
  26. 2023Leveraging Jumpy Models for Planning and Fast Learning in Robotic Domainscited by 6
  27. 2024Effective Communication with Dynamic Feature Compressioncited by 6
  28. 2026RAPO: Expanding Exploration for LLM Agents via Retrieval-Augmented Policy Optimizationcited by 6
  29. 2022Information-Theoretic Odometry Learningcited by 5
  30. 2022Rate-Constrained Remote Contextual Banditscited by 5
  31. 2023Approximate information state based convergence analysis of recurrent Q-learningcited by 5
  32. 2025SOE: Sample-Efficient Robot Policy Self-Improvement via On-Manifold Explorationcited by 5
  33. 2025Information-Theoretic Reward Modeling for Stable RLHF: Detecting and Mitigating Reward Hackingcited by 5
  34. 2023Bayesian Reinforcement Learning with Limited Cognitive Loadcited by 4
  35. 2023Offline Diversity Maximization Under Imitation Constraintscited by 4
  36. 2024Satisficing Exploration for Deep Reinforcement Learningcited by 4
  37. 2022Priors, Hierarchy, and Information Asymmetry for Skill Transfer in Reinforcement Learningcited by 3
  38. 2023Semantic and Effective Communication for Remote Control Tasks with Dynamic Feature Compressioncited by 3

50 most cited of 78 All 78 (JSON)