mlsanityData to 7 Oct 2026

Neural Distillation as a State Representation Bottleneck in Reinforcement Learning

arXiv 2210.02224 ↗First posted 5 Oct 2022cs.LG cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Learning a good state representation is a critical skill when dealing with multiple tasks in Reinforcement Learning as it allows for transfer and better generalization between tasks. However, defining what constitute a useful representation is far from simple and there is so far no standard method to find such an encoding. In this paper, we argue that distillation -- a process that aims at imitating a set of given policies with a single neural network -- can be used to learn a state representation displaying favorable characteristics. In this regard, we define three criteria that measure desirable features of a state encoding: the ability to select important variables in the input space, the ability to efficiently separate states according to their corresponding optimal action, and the robustness of the state encoding on new tasks. We first evaluate these criteria and verify the contribution of distillation on state representation on a toy environment based on the standard inverted pendulum problem, before extending our analysis on more complex visual tasks from the Atari and Procgen benchmarks.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 42 of its 62 references that are in the map. First posted on arXiv 5 Oct 2022.

Builds on

42 of 62 references.

References · 42

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  4. 2016Progressive Neural Networkscited by 1,332
  5. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  6. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  7. 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
  8. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  9. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  10. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  11. 2019Leveraging Procedural Generation to Benchmark Reinforcement Learningcited by 478
  12. 2020Reinforcement Learning with Augmented Datacited by 452
30 more
  1. 2015Policy Distillationcited by 414
  2. 2020Multi-Task Learning with Deep Neural Networks: A Surveycited by 374
  3. 2017PathNet: Evolution Channels Gradient Descent in Super Neural Networkscited by 362
  4. 2018Quantifying Generalization in Reinforcement Learningcited by 345
  5. 2017Implicit Regularization in Matrix Factorizationcited by 318
  6. 2017Distral: Robust Multitask Reinforcement Learningcited by 231
  7. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
  8. 2015Actor-Mimic: Deep Multitask and Transfer Reinforcement Learningcited by 176
  9. 2018A Study on Overfitting in Deep Reinforcement Learningcited by 170
  10. 2021Multi-Task Reinforcement Learning with Context-based Representationscited by 169
  11. 2017Visualizing and Understanding Atari Agentscited by 141
  12. 2018Multi-task Deep Reinforcement Learning with PopArtcited by 138
  13. 2018Assessing Generalization in Deep Reinforcement Learningcited by 132
  14. 2018State Representation Learning for Control: An Overviewcited by 125
  15. 2018Dopamine: A Research Framework for Deep Reinforcement Learningcited by 121
  16. 2018Generalization and Regularization in DQNcited by 117
  17. 2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
  18. 2019Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learningcited by 102
  19. 2019Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneckcited by 101
  20. 2019Observational Overfitting in Reinforcement Learningcited by 90
  21. 2020Improving Generalization in Reinforcement Learning with Mixture Regularizationcited by 81
  22. 2021Decoupling Value and Policy for Generalization in Reinforcement Learningcited by 81
  23. 2020Automatic Data Augmentation for Generalization in Deep Reinforcement Learningcited by 71
  24. 2016Graying the black box: Understanding DQNscited by 68
  25. 2019A Geometric Perspective on Optimal Representations for Reinforcement Learningcited by 54
  26. 2019Explain Your Move: Understanding Agent Actions Using Specific and Relevant Feature Attributioncited by 50
  27. 2020Local and Global Explanations of Agent Behavior: Integrating Strategy Summaries with Saliency Mapscited by 30
  28. 2019Decoupling feature extraction from policy learning: assessing benefits of state representation learning in goal based roboticscited by 21
  29. 2022Local Feature Swapping for Generalization in Reinforcement Learningcited by 9
  30. 2019S-TRIGGER: Continual State Representation Learning via Self-Triggered Generative Replaycited by 7

Built on by

Cited by year

Citing papers per year: 2023: 1.

1 paper.

  1. 2023Prediction and Control in Continual Reinforcement Learningcited by 19