Overcoming classic challenges for artificial neural networks by providing incentives and practice
Not on the map. Why?
Abstract (arXiv)
Since the earliest proposals for artificial neural network (ANN) models of the mind and brain, critics have pointed out key weaknesses in these models compared to human cognitive abilities. Here we review recent work that uses metalearning to overcome several classic challenges, which we characterize as addressing the Problem of Incentive and Practice -- that is, providing machines with both incentives to improve specific skills and opportunities to practice those skills. This explicit optimization contrasts with more conventional approaches that hope the desired behaviour will emerge through optimizing related but different objectives. We review applications of this principle to addressing four classic challenges for ANNs: systematic generalization, catastrophic forgetting, few-shot learning and multi-step reasoning. We also discuss how large language models incorporate key aspects of this metalearning framework (namely, sequence prediction with feedback trained on diverse data), which helps to explain some of their successes on these classic challenges. Finally, we discuss the prospects for understanding aspects of human development through this framework, and whether natural environments provide the right incentives and practice for learning how to make challenging generalizations.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 65 of its 162 references that are in the map. First posted on arXiv 14 Oct 2024.
Builds on
65 of 162 references.
References · 65
- 2017Attention Is All You Needcited by 59,932
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
- 2023Let's Verify Step by Stepcited by 3,598
- 2017Prototypical Networks for Few-shot Learningcited by 3,001
- 2022Emergent Abilities of Large Language Modelscited by 2,483
- 2016Matching Networks for One Shot Learningcited by 2,246
38 more
- 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
- 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
- 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
- 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
- 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
- 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
- 2014Deep Learning in Neural Networks: An Overviewcited by 765
- 2022Transformers learn in-context by gradient descentcited by 714
- 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 700
- 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
- 2019Three scenarios for continual learningcited by 610
- 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
- 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 471
- 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
- 2016Learning to reinforcement learncited by 369
- 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
- 2022Data Distributional Properties Drive Emergent In-Context Learning in Transformerscited by 303
- 2019Measuring Compositional Generalization: A Comprehensive Method on Realistic Datacited by 280
- 2017Meta Networkscited by 268
- 2019Compositionality decomposed: how do neural networks generalise?cited by 264
- 2020COGS: A Compositional Generalization Challenge Based on Semantic Interpretationcited by 260
- 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 250
- 2020On the Binding Problem in Artificial Neural Networkscited by 230
- 2018Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselinescited by 228
- 2019Meta-Learning Representations for Continual Learningcited by 189
- 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 175
- 2024Stream of Search (SoS): Learning to Search in Languagecited by 141
- 2023Pretraining task diversity and the emergence of non-Bayesian in-context learning for regressioncited by 138
- 2021The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformerscited by 117
- 2021Going Beyond Linear Transformers with Recurrent Fast Weight Programmerscited by 84
- 2020Learning to Continually Learncited by 82
- 2020Compositional Generalization via Neural-Symbolic Stack Machinescited by 80
- 2024Prover-Verifier Games improve legibility of LLM outputscited by 79
- 2023In-Context Learning through the Bayesian Prismcited by 78
- 2020Learning Compositional Rules via Neural Program Synthesiscited by 74
- 2017Meta-Learning and Universality: Deep Representations and Gradient Descent can Approximate any Learning Algorithmcited by 65
- 2019CLOSURE: Assessing Systematic Generalization of CLEVR Modelscited by 63
- 2021Meta-Learning to Compositionally Generalizecited by 62
50 most cited of 65 All 65 (JSON)
Built on by
Citing papers per year: 2025: 3.
3 papers.