mlsanityData to 7 Oct 2026

Predictive Information Accelerates Learning in RL

arXiv 2007.12401 ↗First posted 24 Jul 2020cs.LG cs.AI cs.ITcited by 56

Not on the map. Why?

Abstract (arXiv)

The Predictive Information is the mutual information between the past and the future, I(X_past; X_future). We hypothesize that capturing the predictive information is useful in RL, since the ability to model what will happen next is necessary for success on many tasks. To test our hypothesis, we train Soft Actor-Critic (SAC) agents from pixels with an auxiliary task that learns a compressed representation of the predictive information of the RL environment dynamics using a contrastive version of the Conditional Entropy Bottleneck (CEB) objective. We refer to these as Predictive Information SAC (PI-SAC) agents. We show that PI-SAC agents can substantially improve sample efficiency over challenging baselines on tasks from the DM Control suite of continuous control environments. We evaluate PI-SAC agents by comparing against uncompressed PI-SAC agents, other compressed and uncompressed agents, and SAC agents directly trained from pixels. Our implementation is given on GitHub.

Key facts. Cited by 56 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 19; 2023: 11; 2024: 5; 2025: 6; 2026: 1.

56 papers.

Sort:

By area:

  1. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  2. 2022Multi-Game Decision Transformerscited by 212
  3. 2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
  4. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 116
  5. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 82
  6. 2022Denoised MDPs: Learning World Models Better Than the World Itselfcited by 66
  7. 2021Learning Task Informed Abstractionscited by 58
  8. 2021Tactical Optimism and Pessimism for Deep Reinforcement Learningcited by 55
  9. 2021PlayVirtual: Augmenting Cycle-Consistent Virtual Trajectories for Reinforcement Learningcited by 41
  10. 2021Compressive Visual Representationscited by 41
  11. 2022Mask-based Latent Reconstruction for Reinforcement Learningcited by 39
  12. 2023Crossway Diffusion: Improving Diffusion-based Visuomotor Policy via Self-supervised Learningcited by 37
38 more
  1. 2021Return-Based Contrastive Representation Learning for Reinforcement Learningcited by 36
  2. 2021DRIBO: Robust Deep Reinforcement Learning via Multi-View Information Bottleneckcited by 31
  3. 2022INFOrmation Prioritization through EmPOWERment in Visual Model-Based RLcited by 27
  4. 2023Revisiting Plasticity in Visual Reinforcement Learning: Data, Modules and Training Stagescited by 27
  5. 2021Learning Representations for Pixel-based Control: What Matters and Why?cited by 24
  6. 2022Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?cited by 24
  7. 2022A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learningcited by 23
  8. 2023Learning World Models with Identifiable Factorizationcited by 23
  9. 2021Test Distribution-Aware Active Learning: A Principled Approach Against Distribution Shift and Outlierscited by 22
  10. 2022Unsupervised Representation Learning in Deep Reinforcement Learning: A Reviewcited by 18
  11. 2021Hyperparameter Selection for Imitation Learningcited by 17
  12. 2022An Empirical Investigation of Representation Learning for Imitationcited by 15
  13. 2021SimSR: Simple Distance-based State Representation for Deep Reinforcement Learningcited by 14
  14. 2023Robust Representation Learning by Clustering with Bisimulation Metrics for Visual Reinforcement Learning with Distractionscited by 13
  15. 2022Integrating Contrastive Learning with Dynamic Models for Reinforcement Learning from Imagescited by 12
  16. 2022Evaluating Vision Transformer Methods for Deep Reinforcement Learning from Pixelscited by 12
  17. 2021Learning Temporally-Consistent Representations for Data-Efficient Reinforcement Learningcited by 10
  18. 2022PI-ARS: Accelerating Evolution-Learned Visual-Locomotion with Predictive Information Representationscited by 10
  19. 2022Structure-Encoding Auxiliary Tasks for Improved Visual Representation in Vision-and-Language Navigationcited by 9
  20. 2023Learning Better with Less: Effective Augmentation for Sample-Efficient Visual Reinforcement Learningcited by 8
  21. 2024Multi-view Disentanglement for Reinforcement Learning with Multiple Camerascited by 8
  22. 2022Multi-Horizon Representations with Hierarchical Forward Models for Reinforcement Learningcited by 7
  23. 2024Advancing Deep Active Learning & Data Subset Selection: Unifying Principles with Information-Theory Intuitionscited by 7
  24. 2025Studying the Interplay Between the Actor and Critic Representations in Reinforcement Learningcited by 7
  25. 2021Learning State Representations from Random Deep Action-conditional Predictionscited by 6
  26. 2022PI-QT-Opt: Predictive Information Improves Multi-Task Robotic Reinforcement Learning at Scalecited by 5
  27. 2025Enhancing Tactile-based Reinforcement Learning for Robotic Controlcited by 5
  28. 2023Local Optimization Achieves Global Optimality in Multi-Agent Reinforcement Learningcited by 4
  29. 2024Multimodal Information Bottleneck for Deep Reinforcement Learning with Multiple Sensorscited by 4
  30. 2022Accelerating Representation Learning with View-Consistent Dynamics in Data-Efficient Reinforcement Learningcited by 3
  31. 2022Learning List-wise Representation in Reinforcement Learning for Ads Allocation with Multiple Auxiliary Taskscited by 3
  32. 2023Sample-efficient Real-time Planning with Curiosity Cross-Entropy Method and Contrastive Learningcited by 3
  33. 2024An Empirical Study on the Power of Future Prediction in Partially Observable Environmentscited by 3
  34. 2024State Chrono Representation for Enhancing Generalization in Reinforcement Learningcited by 3
  35. 2022Disentangled (Un)Controllable Featurescited by 2
  36. 2025From Pixels to Factors: Learning Independently Controllable State Variables for Reinforcement Learningcited by 2
  37. 2023Deep Occupancy-Predictive Representations for Autonomous Drivingcited by 1
  38. 2023Visual Forecasting as a Mid-level Representation for Avoidancecited by 1

50 most cited of 56 All 56 (JSON)