mlsanityData to 7 Oct 2026

Representation Convergence: Mutual Distillation is Secretly a Form of Regularization

arXiv 2501.02481 ↗First posted 5 Jan 2025cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

In this paper, we argue that mutual distillation between reinforcement learning policies serves as an implicit regularization, preventing them from overfitting to irrelevant features. We highlight two separate contributions: (i) Theoretically, for the first time, we prove that enhancing the policy robustness to irrelevant features leads to improved generalization performance. (ii) Empirically, we demonstrate that mutual distillation between policies contributes to such robustness, enabling the spontaneous emergence of invariant representations over pixel inputs. Ultimately, we do not claim to achieve state-of-the-art performance but rather focus on uncovering the underlying principles of generalization and deepening our understanding of its mechanisms.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 39 of its 54 references that are in the map. First posted on arXiv 5 Jan 2025.

Builds on

39 of 54 references.

References · 39

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  5. 2017Constrained Policy Optimizationcited by 928
  6. 2017Deep Mutual Learningcited by 621
  7. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  8. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  9. 2020Reinforcement Learning with Augmented Datacited by 452
  10. 2024The Platonic Representation Hypothesiscited by 362
  11. 2018Quantifying Generalization in Reinforcement Learningcited by 345
  12. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
27 more
  1. 2019Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization without Accessing Target Domain Datacited by 245
  2. 2018A Study on Overfitting in Deep Reinforcement Learningcited by 170
  3. 2019Active Domain Randomizationcited by 161
  4. 2018Assessing Generalization in Deep Reinforcement Learningcited by 132
  5. 2020Phasic Policy Gradientcited by 121
  6. 2018Generalization and Regularization in DQNcited by 117
  7. 2019Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learningcited by 102
  8. 2019Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneckcited by 101
  9. 2019Observational Overfitting in Reinforcement Learningcited by 90
  10. 2020Improving Generalization in Reinforcement Learning with Mixture Regularizationcited by 81
  11. 2021Decoupling Value and Policy for Generalization in Reinforcement Learningcited by 81
  12. 2018Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generationcited by 80
  13. 2017Towards Generalization and Simplicity in Continuous Controlcited by 65
  14. 2021Domain Invariant Representation Learning with Domain Density Transformationscited by 53
  15. 2023Behavior Proximal Policy Optimizationcited by 43
  16. 2024Simple Policy Optimizationcited by 31
  17. 2023RL-ViGen: A Reinforcement Learning Benchmark for Visual Generalizationcited by 27
  18. 2022A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learningcited by 23
  19. 2020Measuring Visual Generalization in Continuous Control from Pixelscited by 20
  20. 2021Generalization of Reinforcement Learning with Policy-Aware Adversarial Data Augmentationcited by 15
  21. 2019Robust Visual Domain Randomization for Reinforcement Learningcited by 13
  22. 2020Robust Domain Randomised Reinforcement Learning through Peer-to-Peer Distillationcited by 10
  23. 2022Rethinking Value Function Learning for Generalization in Reinforcement Learningcited by 8
  24. 2024Improving Generalization on the ProcGen Benchmark with Simple Architectural Changes and Scalecited by 6
  25. 2024Transductive Off-policy Proximal Policy Optimizationcited by 5
  26. 2023Adversarial Style Transfer for Robust Policy Optimization in Deep Reinforcement Learningcited by 2
  27. 2024Twin Network Augmentation: A Novel Training Strategy for Improved Spiking Neural Networks and Efficient Weight Quantizationcited by 1

Built on by

Not cited yet.