mlsanityData to 14 Oct 2026

What Neuroscience Can Teach AI About Learning in Continuously Changing Environments

arXiv 2507.02103 ↗First posted 2 Jul 2025cs.AI q-bio.NCcited by 0

Not on the map. Why?

Abstract (arXiv)

Modern AI models, such as large language models, are usually trained once on a huge corpus of data, potentially fine-tuned for a specific task, and then deployed with fixed parameters. Their training is costly, slow, and gradual, requiring billions of repetitions. In stark contrast, animals continuously adapt to the ever-changing contingencies in their environments. This is particularly important for social species, where behavioral policies and reward outcomes may frequently change in interaction with peers. The underlying computational processes are often marked by rapid shifts in an animal's behaviour and rather sudden transitions in neuronal population activity. Such computational capacities are of growing importance for AI systems operating in the real world, like those guiding robots or autonomous vehicles, or for agentic AI interacting with humans online. Can AI learn from neuroscience? This Perspective explores this question, integrating the literature on continual and in-context learning in AI with the neuroscience of learning on behavioral tasks with shifting rules, reward probabilities, or outcomes. We will outline an agenda for how specifically insights from neuroscience may inform current developments in AI in this area, and - vice versa - what neuroscience may learn from AI, contributing to the evolving field of NeuroAI.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 59 of its 163 references that are in the map. First posted on arXiv 2 Jul 2025.

Builds on

59 of 163 references.

References · 59

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  7. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  8. 2016Learning without Forgettingcited by 2,496
  9. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  10. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  11. 2017Continual Learning with Deep Generative Replaycited by 1,158
  12. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
38 more
  1. 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 772
  2. 2024Chronos: Learning the Language of Time Seriescited by 759
  3. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 742
  4. 2022Transformers learn in-context by gradient descentcited by 732
  5. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 722
  6. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 715
  7. 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 521
  8. 2017Variational Continual Learningcited by 339
  9. 2022Data Distributional Properties Drive Emergent In-Context Learning in Transformerscited by 309
  10. 2023Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selectioncited by 298
  11. 2023Transformers as Algorithms: Generalization and Stability in In-context Learningcited by 222
  12. 2022Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizerscited by 204
  13. 2023RanPAC: Random Projections and Pre-trained Models for Continual Learningcited by 164
  14. 2023A mathematical perspective on Transformerscited by 164
  15. 2023A Theory of Emergent In-Context Learning as Implicit Structure Inductioncited by 86
  16. 2023The Transient Nature of Emergent In-Context Learning in Transformerscited by 76
  17. 2021On the difficulty of learning chaotic dynamics with RNNscited by 67
  18. 2024Towards Lifelong Learning of Large Language Models: A Surveycited by 64
  19. 2021Continual Learning via Local Module Compositioncited by 63
  20. 2020Hausdorff Dimension, Heavy Tails, and Generalization in Neural Networkscited by 62
  21. 2021Attractor and integrator networks in the braincited by 61
  22. 2023Generalized Teacher Forcing for Learning Chaotic Dynamicscited by 58
  23. 2025Training Dynamics of In-Context Learning in Linear Attentioncited by 46
  24. 2023Pretraining Data Mixtures Enable Narrow Model Selection Capabilities in Transformer Modelscited by 45
  25. 2024Is Mamba Capable of In-Context Learning?cited by 45
  26. 2023In search of dispersed memories: Generative diffusion models are associative memory networkscited by 44
  27. 2022Modular Lifelong Reinforcement Learning via Neural Compositioncited by 40
  28. 2025Memorization to Generalization: Emergence of Diffusion Models from Associative Memorycited by 36
  29. 2024The broader spectrum of in-context learningcited by 35
  30. 2018Sparse Attentive Backtracking: Temporal CreditAssignment Through Remindingcited by 34
  31. 2022Using Machine Learning to Anticipate Tipping Points and Extrapolate to Post-Tipping Dynamics of Non-Stationary Dynamical Systemscited by 33
  32. 2024Out-of-Domain Generalization in Dynamical Systems Reconstructioncited by 33
  33. 2018Continual Reinforcement Learning with Complex Synapsescited by 32
  34. 2021Synaptic metaplasticity in binarized neural networkscited by 29
  35. 2023In-context Learning and Gradient Descent Revisitedcited by 20
  36. 2024A Unified and General Framework for Continual Learningcited by 20
  37. 2023Bifurcations and loss jumps in RNN trainingcited by 17
  38. 2024Mamba4Cast: Efficient Zero-Shot Time Series Forecasting with State Space Modelscited by 17

50 most cited of 59 All 59 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Adaptive Intelligence: leveraging insights from adaptive behavior in animals to build flexible AI systems
  2. 2016Discovering Latent States for Model Learning: Applying Sensorimotor Contingencies Theory and Predictive Processing to Model Context
  3. 2021A brain basis of dynamical intelligence for AI and computational neuroscience
  4. 2024Lifelong Reinforcement Learning via Neuromodulation
  5. 2025Behavioral Exploration: Learning to Explore via In-Context Adaptation
  6. 2024Deep reinforcement learning with time-scale invariant memory
  7. 2025Flexible inference for animal learning rules using neural networks
  8. 2026How much neuroscience does a neuroscientist need to know? ↗