mlsanityData to 14 Oct 2026

Maximum Total Correlation Reinforcement Learning

arXiv 2505.16734 ↗First posted 22 May 2025cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Simplicity is a powerful inductive bias. In reinforcement learning, regularization is used for simpler policies, data augmentation for simpler representations, and sparse reward functions for simpler objectives, all that, with the underlying motivation to increase generalizability and robustness by focusing on the essentials. Supplementary to these techniques, we investigate how to promote simple behavior throughout the episode. To that end, we introduce a modification of the reinforcement learning problem that additionally maximizes the total correlation within the induced trajectories. We propose a practical algorithm that optimizes all models, including policy and state representation, based on a lower-bound approximation. In simulated robot environments, our method naturally generates policies that induce periodic and compressible trajectories, and that exhibit superior robustness to noise and changes in dynamics compared to baseline methods, while also improving performance in the original tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 38 of its 51 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

38 of 51 references.

References · 38

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  3. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  4. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  5. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  6. 2018DeepMind Control Suitecited by 956
  7. 2016Deep Variational Information Bottleneckcited by 930
  8. 2015Deep Learning and the Information Bottleneck Principlecited by 802
  9. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  10. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  11. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
  12. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 274
26 more
  1. 2023Real-World Humanoid Locomotion with Reinforcement Learningcited by 187
  2. 2019Action Robust Reinforcement Learning and Applications in Continuous Controlcited by 174
  3. 2019Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneckcited by 101
  4. 2022Distributionally Robust Model-Based Offline Reinforcement Learning with Near-Optimal Sample Complexitycited by 87
  5. 2020Deep Reinforcement and InfoMax Learningcited by 79
  6. 2019InfoBot: Transfer and Exploration via the Information Bottleneckcited by 78
  7. 2022BYOL-Explore: Exploration by Bootstrapped Predictioncited by 73
  8. 2020Predictive Information Accelerates Learning in RLcited by 56
  9. 2021Which Mutual-Information Representation Learning Objectives are Sufficient for Control?cited by 29
  10. 2021Robust Predictable Controlcited by 29
  11. 2023Conditional Mutual Information for Disentangled Representations in Reinforcement Learningcited by 27
  12. 2020Dynamics Generalization via Information Bottleneck in Deep Reinforcement Learningcited by 26
  13. 2021Dynamic Bottleneck for Robust Self-Supervised Explorationcited by 21
  14. 2019Mutual-Information Regularization in Markov Decision Processes and Actor-Critic Learningcited by 18
  15. 2023Context Shift Reduction for Offline Meta-Reinforcement Learningcited by 18
  16. 2018Auto-Encoding Total Correlation Explanationcited by 17
  17. 2021Functional Regularization for Reinforcement Learning via Learned Fourier Featurescited by 17
  18. 2022Mutual Information Regularized Offline Reinforcement Learningcited by 12
  19. 2022Max-Min Off-Policy Actor-Critic Method Focusing on Worst-Case Robustness to Model Misspecificationcited by 11
  20. 2023Reinforcement Learning with Simple Sequence Priorscited by 11
  21. 2023Robust Adversarial Reinforcement Learning via Bounded Rationality Curriculacited by 10
  22. 2023State Sequences Prediction via Fourier Transform for Representation Learningcited by 7
  23. 2014Discovering Structure in High-Dimensional Data Through Correlation Explanationcited by 5
  24. 2023STEERING: Stein Information Directed Exploration for Model-Based Reinforcement Learningcited by 4
  25. 2017Unsupervised Learning via Total Correlation Explanationcited by 3
  26. 2022Dimensionality Reduction and Prioritized Exploration for Policy Searchcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Trajectory Entropy Reinforcement Learning for Robust Robot Motor Skill Learning
  2. 2024Improving generalization of robot locomotion policies via Sharpness-Aware Reinforcement Learning
  3. 2026Redesigning Regularization for Effective Policy Smoothing
  4. 2023Reinforcement Learning with General Utilities: Simpler Variance Reduction and Large State-Action Space
  5. 2026Revisiting Temporal Regularization for Smooth Control in Deep Reinforcement Learning
  6. 2025Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics
  7. 2026Physics-Informed Policy Optimization via Analytic Dynamics Regularization
  8. 2026TMRL: Diffusion Timestep-Modulated Pretraining Enables Exploration for Efficient Policy Finetuning