From Prediction to Understanding: Will AI Foundation Models Transform Brain Science?
Not on the map. Why?
Abstract (arXiv)
Generative pretraining (the "GPT" in ChatGPT) enables language models to learn from vast amounts of internet text without human supervision. This approach has driven breakthroughs across AI by allowing deep neural networks to learn from massive, unstructured datasets. We use the term foundation models to refer to large pretrained systems that can be adapted to a wide range of tasks within and across domains, and these models are increasingly applied beyond language to the brain sciences. These models achieve strong predictive accuracy, raising hopes that they might illuminate computational principles. But predictive success alone does not guarantee scientific understanding. Here, we outline how foundation models can be productively integrated into the brain sciences, highlighting both their promise and their limitations. The central challenge is to move from prediction to explanation: linking model computations to mechanisms underlying neural activity and cognition.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 31 of its 65 references that are in the map. First posted on arXiv 21 Sep 2025.
Builds on
31 of 65 references.
References · 31
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
- 2017Deep reinforcement learning from human preferencescited by 3,846
19 more
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2020Shortcut Learning in Deep Neural Networkscited by 1,928
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
- 2023Steering Language Models With Activation Engineeringcited by 824
- 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 512
- 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
- 2023Function Vectors in Large Language Modelscited by 292
- 2021Thinking Like Transformerscited by 182
- 2024Emergent World Models and Latent Variable Estimation in Chess-Playing Language Modelscited by 60
- 2025What Has a Foundation Model Found? Using Inductive Bias to Probe for World Modelscited by 46
- 2021What if This Modified That? Syntactic Interventions via Counterfactual Embeddingscited by 39
- 2023Understanding Addition in Transformerscited by 34
- 2024Political-LLM: Large Language Models in Political Sciencecited by 27
- 2020Recurrent neural circuits for contour detectioncited by 23
- 2024LLMs as Models for Analogical Reasoningcited by 16
- 2025Towards a general-purpose foundation model for fMRI analysiscited by 15
- 2025Better artificial intelligence does not mean better models of biologycited by 10
- 2024A Large-Scale Simulation on Large Language Models for Decision-Making in Political Sciencecited by 9
- 2025Not Yet AlphaFold for the Mind: Evaluating Centaur as a Synthetic Participantcited by 4
Built on by
Citing papers per year: 2026: 2.
2 papers.