Reinforcement Teaching
Not on the map. Why?
Abstract (arXiv)
Machine learning algorithms learn to solve a task, but are unable to improve their ability to learn. Meta-learning methods learn about machine learning algorithms and improve them so that they learn more quickly. However, existing meta-learning methods are either hand-crafted to improve one specific component of an algorithm or only work with differentiable algorithms. We develop a unifying meta-learning framework, called Reinforcement Teaching, to improve the learning process of any algorithm. Under Reinforcement Teaching, a teaching policy is learned, through reinforcement, to improve a student's learning algorithm. To learn an effective teaching policy, we introduce the parametric-behavior embedder that learns a representation of the student's learnable parameters from its input/output behavior. We further use learning progress to shape the teacher's reward, allowing it to more quickly maximize the student's performance. To demonstrate the generality of Reinforcement Teaching, we conduct experiments in which a teacher learns to significantly improve both reinforcement and supervised learning algorithms. Reinforcement Teaching outperforms previous work using heuristic reward functions and state representations, as well as other parameter representations.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 62 of its 113 references that are in the map. First posted on arXiv 25 Apr 2022.
Builds on
62 of 113 references.
References · 62
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
- 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
- 2015Continuous control with deep reinforcement learningcited by 3,407
- 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
- 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
- 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
- 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
- 2017Deep Setscited by 1,421
38 more
- 2020Meta-Learning in Neural Networks: A Surveycited by 950
- 2021A Minimalist Approach to Offline Reinforcement Learningcited by 899
- 2019Behavior Regularized Offline Reinforcement Learningcited by 662
- 2020MOPO: Model-based Offline Policy Optimizationcited by 636
- 2016Learning to learn by gradient descent by gradient descentcited by 620
- 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
- 2016Learning to reinforcement learncited by 377
- 2015Gradient-based Hyperparameter Optimization through Reversible Learningcited by 360
- 2017Stochastic Gradient Descent as Approximate Bayesian Inferencecited by 300
- 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
- 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 260
- 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
- 2017Automated Curriculum Learning for Neural Networkscited by 257
- 2017Teacher-Student Curriculum Learningcited by 210
- 2020Prioritized Level Replaycited by 198
- 2016Learning to Optimizecited by 195
- 2019Meta-Learning Representations for Continual Learningcited by 189
- 2022Evolving Curricula with Regret-Based Environment Designcited by 173
- 2018Meta-Gradient Reinforcement Learningcited by 159
- 2019Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environmentscited by 119
- 2020Effective Diversity in Population Based Reinforcement Learningcited by 119
- 2021Replay-Guided Adversarial Environment Designcited by 111
- 2019Meta-Learning with Warped Gradient Descentcited by 98
- 2017Bridging the Gap between Constant Step Size Stochastic Gradient Descent and Markov Chainscited by 88
- 2020Learning with AMIGo: Adversarially Motivated Intrinsic Goalscited by 88
- 2020Adaptive Reward-Poisoning Attacks against Reinforcement Learningcited by 86
- 2018Understanding Short-Horizon Bias in Stochastic Meta-Optimizationcited by 69
- 2019Large Scale Structure of Neural Network Loss Landscapescited by 68
- 2018An Overview of Machine Teachingcited by 63
- 2019Weight-space symmetry in deep networks gives rise to permutation saddles, connected by equal-loss valleys across the loss landscapecited by 63
- 2020Learning and Planning in Average-Reward Markov Decision Processescited by 54
- 2021The Sensory Neuron as a Transformer: Permutation-Invariant Neural Networks for Reinforcement Learningcited by 50
- 2021Bootstrapped Meta-Learningcited by 45
- 2018Learning To Simulatecited by 44
- 2017Learning to Optimize Neural Netscited by 41
- 2018Learning to Teach with Dynamic Loss Functionscited by 40
- 2019Hyp-RL : Hyperparameter Optimization by Reinforcement Learningcited by 36
- 2018Learning Curriculum Policies for Reinforcement Learningcited by 35
50 most cited of 62 All 62 (JSON)
Built on by
Citing papers per year: 2024: 1.
1 paper.