mlsanityData to 7 Oct 2026

Meta-Learning Strategies through Value Maximization in Neural Networks

arXiv 2310.19919 ↗First posted 30 Oct 2023cs.NE cs.LG q-bio.NCcited by 4

Not on the map. Why?

Abstract (arXiv)

Biological and artificial learning agents face numerous choices about how to learn, ranging from hyperparameter selection to aspects of task distributions like curricula. Understanding how to make these meta-learning choices could offer normative accounts of cognitive control functions in biological learners and improve engineered systems. Yet optimal strategies remain challenging to compute in modern deep networks due to the complexity of optimizing through the entire learning process. Here we theoretically investigate optimal strategies in a tractable setting. We present a learning effort framework capable of efficiently optimizing control signals on a fully normative objective: discounted cumulative performance throughout learning. We obtain computational tractability by using average dynamical equations for gradient descent, available for simple neural network architectures. Our framework accommodates a range of meta-learning and automatic curriculum learning methods in a unified normative setting. We apply this framework to investigate the effect of approximations in common meta-learning algorithms; infer aspects of optimal curricula; and compute optimal neuronal resource allocation in a continual learning setting. Across settings, we find that control effort is most beneficial when applied to easier aspects of a task early in learning; followed by sustained effort on harder aspects. Overall, the learning effort framework provides a tractable theoretical test bed to study normative benefits of interventions in a variety of learning systems, as well as a formal account of optimal cognitive control strategies over learning trajectories posited by established theories in cognitive neuroscience.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 33 of its 84 references that are in the map. First posted on arXiv 30 Oct 2023.

Builds on

33 of 84 references.

References · 33

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  3. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,313
  4. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  5. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  6. 2018On Lazy Training in Differentiable Programmingcited by 671
  7. 2019Meta-Learning with Implicit Gradientscited by 523
  8. 2018Bilevel Programming for Hyperparameter Optimization and Meta-Learningcited by 403
  9. 2015Neural Module Networkscited by 375
  10. 2016Learning to reinforcement learncited by 369
  11. 2018A mathematical theory of semantic development in deep neural networkscited by 182
  12. 2017Neural Episodic Controlcited by 148
21 more
  1. 2018Meta-Reinforcement Learning of Structured Exploration Strategiescited by 128
  2. 2019Dynamics of stochastic gradient descent for two-layer neural networks in the teacher-student setupcited by 112
  3. 2022Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networkscited by 104
  4. 2020Meta-Learning with Adaptive Hyperparameterscited by 79
  5. 2020Revisiting Rainbow: Promoting more Insightful and Inclusive Deep Reinforcement Learning Researchcited by 78
  6. 2020Statistical Mechanics of Deep Linear Neural Networks: The Back-Propagating Kernel Renormalizationcited by 71
  7. 2020Dynamical mean-field theory for stochastic gradient descent in Gaussian mixture classificationcited by 60
  8. 2020Decoupling Exploration and Exploitation for Meta-Reinforcement Learning without Sacrificescited by 59
  9. 2022The Neural Race Reduction: Dynamics of Abstraction in Gated Networkscited by 37
  10. 2021Continuous vs. Discrete Optimization of Deep Neural Networkscited by 34
  11. 2021An Analytical Theory of Curriculum Learning in Teacher-Student Networkscited by 27
  12. 2018Been There, Done That: Meta-Learning with Episodic Recallcited by 25
  13. 2021Lifelong Teacher-Student Network Learningcited by 18
  14. 2023Memory-Based Meta-Learning on Non-Stationary Distributionscited by 14
  15. 2022Continuous-Time Meta-Learning with Forward Mode Differentiationcited by 12
  16. 2021Context Meta-Reinforcement Learning via Neuromodulationcited by 11
  17. 2018Coding with transient trajectories in recurrent neural networkscited by 7
  18. 2020Efficiency of learning vs. processing: Towards a normative theory of multitaskingcited by 3
  19. 2021Distilling Reinforcement Learning Tricks for Video Gamescited by 3
  20. 2020Navigating the Trade-Off between Multi-Task Learning and Learning to Multitask in Deep Neural Networkscited by 2
  21. 2021Curriculum Meta-Learning for Few-shot Classificationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 2.

4 papers.

  1. 2024Optimal Protocols for Continual Learning via Statistical Physics and Control Theorycited by 14
  2. 2026Theory of Optimal Learning Rate Schedules and Scaling Laws for a Random Feature Modelcited by 4
  3. 2025A statistical physics framework for optimal learningcited by 3
  4. 2026Optimal Learning Rate Schedule for Balancing Effort and Performancecited by 1