A Theoretical Survey on Foundation Models
Not on the map. Why?
Abstract (arXiv)
Understanding the inner mechanisms of black-box foundation models (FMs) is essential yet challenging in artificial intelligence and its applications. Over the last decade, the long-running focus has been on their explainability, leading to the development of post-hoc explainable methods to rationalize the specific decisions already made by black-box FMs. However, these explainable methods have certain limitations in terms of faithfulness and resource requirement. Consequently, a new class of interpretable methods should be considered to unveil the underlying mechanisms of FMs in an accurate, comprehensive, heuristic, and resource-light way. This survey aims to review those interpretable methods that comply with the aforementioned principles and have been successfully applied to FMs. These methods are deeply rooted in machine learning theory, covering the analysis of generalization performance, expressive capability, and dynamic behavior. They provide a thorough interpretation of the entire workflow of FMs, ranging from the inference capability and training dynamics to their ethical implications. Ultimately, drawing upon these interpretations, this review identifies the next frontier research directions for FMs.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 68 of its 180 references that are in the map. First posted on arXiv 15 Oct 2024.
Builds on
68 of 180 references.
References · 68
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
- 2022Emergent Abilities of Large Language Modelscited by 2,512
- 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
- 2022Survey of Hallucination in Natural Language Generationcited by 2,075
- 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
- 2019Language Models as Knowledge Bases?cited by 1,936
- 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
- 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
38 more
- 2020Quantifying Attention Flow in Transformerscited by 775
- 2022A Survey on In-context Learningcited by 721
- 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
- 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 602
- 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 434
- 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 401
- 2021Differentially Private Fine-tuning of Language Modelscited by 400
- 2023A Survey of Hallucination in Large Foundation Modelscited by 368
- 2024A Survey on Hallucination in Large Vision-Language Modelscited by 357
- 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 332
- 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 234
- 2023Nash Learning from Human Feedbackcited by 224
- 2022Why Can GPT Learn In-Context? Language Models Implicitly Perform Gradient Descent as Meta-Optimizerscited by 204
- 2022Will we run out of data? Limits of LLM scaling based on human-generated datacited by 195
- 2023One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attentioncited by 170
- 2022A Review on Language Models as Knowledge Basescited by 157
- 2023LM vs LM: Detecting Factual Errors via Cross Examinationcited by 156
- 2021RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Modelscited by 149
- 2023What and How does In-Context Learning Learn? Bayesian Model Averaging, Parameterization, and Generalizationcited by 118
- 2024How Transformers Learn Causal Structure with Gradient Descentcited by 118
- 2023A Theory for Emergence of Complex Skills in Language Modelscited by 113
- 2024Rethinking Interpretability in the Era of Large Language Modelscited by 111
- 2020Explaining NLP Models via Minimal Contrastive Editing (MiCE)cited by 106
- 2022Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?cited by 103
- 2023Is RLHF More Difficult than Standard RL?cited by 96
- 2023How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?cited by 96
- 2023A Survey on Fairness in Large Language Modelscited by 95
- 2023Transformers as Support Vector Machinescited by 74
- 2020Privacy in Deep Learning: A Surveycited by 71
- 2019Optimization for deep learning: theory and algorithmscited by 70
- 2023Reinforcement Learning with Human Feedback: Learning Dynamic Choices via Pessimismcited by 69
- 2021On the Pitfalls of Analyzing Individual Neurons in Language Modelscited by 62
- 2023JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attentioncited by 52
- 2024From Understanding to Utilization: A Survey on Explainability for Large Language Modelscited by 48
- 2023Training Data Extraction From Pre-trained Language Models: A Surveycited by 45
- 2024Reinforcement Learning from Human Feedback with Active Queriescited by 43
- 2023The Closeness of In-Context Learning and Weight Shifting for Softmax Regressioncited by 41
- 2023GPT-4 Can't Reasoncited by 40
50 most cited of 68 All 68 (JSON)
Built on by
Citing papers per year: 2025: 1.
1 paper.