mlsanityData to 14 Oct 2026

Conditional Mutual Information for Disentangled Representations in Reinforcement Learning

arXiv 2305.14133 ↗First posted 23 May 2023cs.LGcited by 27

Not on the map. Why?

Abstract (arXiv)

Reinforcement Learning (RL) environments can produce training data with spurious correlations between features due to the amount of training data or its limited feature coverage. This can lead to RL agents encoding these misleading correlations in their latent representation, preventing the agent from generalising if the correlation changes within the environment or when deployed in the real world. Disentangled representations can improve robustness, but existing disentanglement techniques that minimise mutual information between features require independent features, thus they cannot disentangle correlated features. We propose an auxiliary task for RL algorithms that learns a disentangled representation of high-dimensional observations with correlated features by minimising the conditional mutual information between features in the representation. We demonstrate experimentally, using continuous control tasks, that our approach improves generalisation under correlation shifts, as well as improving the training performance of RL algorithms in the presence of correlated features.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 38 of its 49 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

38 of 49 references.

References · 38

  1. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  4. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 941
  5. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  6. 2018Disentangling by Factorisingcited by 616
  7. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  8. 2020Captum: A unified and generic model interpretability library for PyTorchcited by 547
  9. 2020Reinforcement Learning with Augmented Datacited by 468
  10. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 436
  11. 2020dm_control: Software and Tasks for Continuous Controlcited by 375
  12. 2018Understanding disentangling in β-VAEcited by 335
26 more
  1. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 273
  2. 2020Weakly-Supervised Disentanglement Without Compromisescited by 239
  3. 2017DARLA: Improving Zero-Shot Transfer in Reinforcement Learningcited by 149
  4. 2020Generalization in Reinforcement Learning by Soft Data Augmentationcited by 137
  5. 2021Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learningcited by 123
  6. 2020Invariant Causal Prediction for Block MDPscited by 105
  7. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 105
  8. 2020On Disentangled Representations Learned From Correlated Datacited by 94
  9. 2019Weakly Supervised Disentanglement with Guaranteescited by 82
  10. 2020Deep Reinforcement and InfoMax Learningcited by 79
  11. 2017Conditional independence testing based on a nearest-neighbor estimator of conditional mutual informationcited by 70
  12. 2021Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICAcited by 64
  13. 2020Causal Curiosity: RL Agents Discovering Self-supervised Experiments for Causal Representation Learningcited by 53
  14. 2023Nonlinear Independent Component Analysis for Principled Disentanglement in Unsupervised Deep Learningcited by 48
  15. 2021Learning Markov State Abstractions for Deep Reinforcement Learningcited by 44
  16. 2019CCMI : Classifier based Conditional Mutual Information Estimationcited by 37
  17. 2021Decoupled Reinforcement Learning to Stabilise Intrinsically-Motivated Explorationcited by 25
  18. 2019Explicit Explore-Exploit Algorithms in Continuous State Spacescited by 19
  19. 2021Domain Adversarial Reinforcement Learningcited by 19
  20. 2022Temporal Disentanglement of Representations for Improved Generalisation in Reinforcement Learningcited by 19
  21. 2021Robust On-Policy Sampling for Data-Efficient Policy Evaluation in Reinforcement Learningcited by 15
  22. 2020C-MI-GAN : Estimation of Conditional Mutual Information using MinMax formulationcited by 10
  23. 2021Disentanglement and Generalization Under Correlation Shiftscited by 8
  24. 2019Exploiting Causality for Selective Belief Filtering in Dynamic Bayesian Networks (Extended Abstract)cited by 7
  25. 2020Neural Estimators for Conditional Mutual Information Using Nearest Neighbors Samplingcited by 7
  26. 2022Invariant Representations with Stochastically Quantized Neural Networkscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 9; 2025: 12; 2026: 4.

27 papers.

Sort:

By area:

  1. 2024emg2pose: A Large and Diverse Benchmark for Surface Electromyographic Hand Pose Estimationcited by 22
  2. 2025A Survey of State Representation Learning for Deep Reinforcement Learningcited by 17
  3. 2024DRED: Zero-Shot Transfer in Reinforcement Learning via Data-Regularised Environment Designcited by 11
  4. 2024Multi-view Disentanglement for Reinforcement Learning with Multiple Camerascited by 8
  5. 2025Studying the Interplay Between the Actor and Critic Representations in Reinforcement Learningcited by 8
  6. 2024DEAR: Disentangled Environment and Agent Representations for Reinforcement Learning without Reconstructioncited by 6
  7. 2025Disentangled World Models: Learning to Transfer Semantic Knowledge from Distracting Videos for Reinforcement Learningcited by 6
  8. 2025Enhancing Tactile-based Reinforcement Learning for Robotic Controlcited by 5
  9. 2024Range, not Independence, Drives Modularity in Biologically Inspired Representationscited by 4
  10. 2025Imagined Autocurriculacited by 4
  11. 2025From Pixels to Factors: Learning Independently Controllable State Variables for Reinforcement Learningcited by 3
  12. 2024Learning Disentangled Representations for Perceptual Point Cloud Quality Assessment via Mutual Information Minimizationcited by 2
15 more
  1. 2024Entropy Regularized Task Representation Learning for Offline Meta-Reinforcement Learningcited by 2
  2. 2025PvP: Data-Efficient Humanoid Robot Learning with Proprioceptive-Privileged Contrastive Representationscited by 2
  3. 2023Contextual Pre-planning on Reward Machine Abstractions for Enhanced Transfer in Deep Reinforcement Learningcited by 1
  4. 2024Skill-aware Mutual Information Optimisation for Generalisation in Reinforcement Learningcited by 1
  5. 2023How the level sampling process impacts zero-shot generalisation in deep reinforcement learningcited by 0
  6. 2024Highway Graph to Accelerate Reinforcement Learningcited by 0
  7. 2025Maximum Total Correlation Reinforcement Learningcited by 0
  8. 2025PTSM: Physiology-aware and Task-invariant Spatio-temporal Modeling for Cross-Subject EEG Decodingcited by 0
  9. 2025Synthetic Data is Sufficient for Zero-Shot Visual Generalization from Offline Datacited by 0
  10. 2025Balanced Multimodal Learning: An Unidirectional Dynamic Interaction Perspectivecited by 0
  11. 2025Inter-Agent Relative Representations for Multi-Agent Option Discoverycited by 0
  12. 2026Sample-wise Constrained Learning via a Sequential Penalty Approach with Applications in Image Processingcited by 0
  13. 2026Conflict-Aware Additive Guidance for Flow Models under Compositional Rewardscited by 0
  14. 2026Coordinated Disentanglement with Iterative Mode Discovery Under Hidden Correlationscited by 0
  15. 2026Divide-and-Remember: Recursive Action-Relevant Memory for Long-Horizon VLA Policiescited by 0