# Emergent Abilities in Large Language Models: A Survey

arXiv: https://arxiv.org/abs/2503.05788 · HTML: https://mlsanity.com/paper/2503.05788/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 99 of its 134 references that are in the map.
- First posted on arXiv 28 Feb 2025.

## Builds on (most cited first, 30 of 99)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://mlsanity.com/paper/2104.08786/) (2021)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [In-context Learning and Induction Heads](https://mlsanity.com/paper/2209.11895/) (2022)
- [An Explanation of In-context Learning as Implicit Bayesian Inference](https://mlsanity.com/paper/2111.02080/) (2021)
- [Show Your Work: Scratchpads for Intermediate Computation with Language Models](https://mlsanity.com/paper/2112.00114/) (2021)
- [ExpeL: LLM Agents Are Experiential Learners](https://mlsanity.com/paper/2308.10144/) (2023)
- [Ignore Previous Prompt: Attack Techniques For Language Models](https://mlsanity.com/paper/2211.09527/) (2022)
- [Large Language Model based Multi-Agents: A Survey of Progress and Challenges](https://mlsanity.com/paper/2402.01680/) (2024)
- [C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Models](https://mlsanity.com/paper/2305.08322/) (2023)

## Built on by (most cited first, 30 of 37)

- [A Survey on Latent Reasoning](https://mlsanity.com/paper/2507.06203/) (2025)
- [BioCLIP 2: Emergent Properties from Scaling Hierarchical Contrastive Learning](https://mlsanity.com/paper/2505.23883/) (2025)
- [MVI-Bench: A Comprehensive Benchmark for Evaluating Robustness to Misleading Visual Inputs in LVLMs](https://mlsanity.com/paper/2511.14159/) (2025)
- [The Illusion of Insight in Reasoning Models](https://mlsanity.com/paper/2601.00514/) (2026)
- [Large Language Models and Emergence: A Complex Systems Perspective](https://mlsanity.com/paper/2506.11135/) (2025)
- [Vision Language Models Cannot Reason About Physical Transformation](https://mlsanity.com/paper/2603.07109/) (2026)
- [From AGI to ASI](https://mlsanity.com/paper/2606.12683/) (2026)
- [Not All Explanations for Deep Learning Phenomena Are Equally Valuable](https://mlsanity.com/paper/2506.23286/) (2025)
- [A Lightweight Modular Framework for Constructing Autonomous Agents Driven by Large Language Models: Design, Implementation, and Applications in AgentForge](https://mlsanity.com/paper/2601.13383/) (2026)
- [Phase Transitions in Attention: A Bayesian Theory of Copy Head Emergence](https://mlsanity.com/paper/2606.12058/) (2026)
- [When Content is Goliath and Algorithm is David: The Style and Semantic Effects of Generative Search Engine](https://mlsanity.com/paper/2509.14436/) (2025)
- [Probing the Hidden Talent of ASR Foundation Models for L2 English Oral Assessment](https://mlsanity.com/paper/2510.16387/) (2025)
- [Mechanism-Based Intelligence (MBI): Differentiable Incentives for Rational Coordination and Guaranteed Alignment in Multi-Agent Systems](https://mlsanity.com/paper/2512.20688/) (2025)
- [Reading Smiles: Proxy Bias in Foundation Models for Facial Emotion Recognition](https://mlsanity.com/paper/2506.19079/) (2025)
- [Why are LLMs' abilities emergent?](https://mlsanity.com/paper/2508.04401/) (2025)
- [Apriel-Nemotron-15B-Thinker](https://mlsanity.com/paper/2508.10948/) (2025)
- [A Review of Developmental Interpretability in Large Language Models](https://mlsanity.com/paper/2508.15841/) (2025)
- [MARS2 2025 Challenge on Multimodal Reasoning: Datasets, Methods, Results, Discussion, and Outlook](https://mlsanity.com/paper/2509.14142/) (2025)
- [Can Small and Reasoning Large Language Models Score Journal Articles for Research Quality and Do Averaging and Few-shot Help?](https://mlsanity.com/paper/2510.22389/) (2025)
- [Semantic Deception: When Reasoning Models Can't Compute an Addition](https://mlsanity.com/paper/2512.20812/) (2025)
- [Shaping Schema via Language Representation as the Next Frontier for LLM Intelligence Expanding](https://mlsanity.com/paper/2605.09271/) (2026)
- [A formal definition and meta-model for a machine theory of mind](https://mlsanity.com/paper/2606.03471/) (2026)
- [Broken Ergodicity and the Violation of the Fluctuation-Dissipation Theorem Lead to Generalization Beyond Overfitting in Machine Learning](https://mlsanity.com/paper/2607.04135/) (2026)
- [Assessing mentalization in humans and large language models](https://mlsanity.com/paper/2608.26291/) (2026)
- [TinyMusician: On-Device Music Generation with Knowledge Distillation and Mixed Precision Quantization](https://mlsanity.com/paper/2509.00914/) (2025)
- [EPT Benchmark: Evaluation of Persian Trustworthiness in Large Language Models](https://mlsanity.com/paper/2509.06838/) (2025)
- [Limits of Emergent Reasoning of Large Language Models in Agentic Frameworks for Deterministic Games](https://mlsanity.com/paper/2510.15974/) (2025)
- [Position: On the Methodological Pitfalls of Evaluating Base LLMs for Reasoning](https://mlsanity.com/paper/2511.10381/) (2025)
- [On the Computability of Artificial General Intelligence](https://mlsanity.com/paper/2512.05212/) (2025)
- [LLM-Enabled Multi-Agent Systems: Empirical Evaluation and Insights into Emerging Design Patterns & Paradigms](https://mlsanity.com/paper/2601.03328/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are leading a new technological revolution as one of the most promising research streams toward artificial general intelligence. The scaling of these models, accomplished by increasing the number of parameters and the magnitude of the training datasets, has been linked to various so-called emergent abilities that were previously unobserved. These emergent abilities, ranging from advanced reasoning and in-context learning to coding and problem-solving, have sparked an intense scientific debate: Are they truly emergent, or do they simply depend on external factors, such as training dynamics, the type of problems, or the chosen metric? What underlying mechanism causes them? Despite their transformative potential, emergent abilities remain poorly understood, leading to misconceptions about their definition, nature, predictability, and implications. In this work, we shed light on emergent abilities by conducting a comprehensive review of the phenomenon, addressing both its scientific underpinnings and real-world consequences. We first critically analyze existing definitions, exposing inconsistencies in conceptualizing emergent abilities. We then explore the conditions under which these abilities appear, evaluating the role of scaling laws, task complexity, pre-training loss, quantization, and prompting strategies. Our review extends beyond traditional LLMs and includes Large Reasoning Models (LRMs), which leverage reinforcement learning and inference-time search to amplify reasoning and self-reflection. However, emergence is not inherently positive. As AI systems gain autonomous reasoning capabilities, they also develop harmful behaviors, including deception, manipulation, and reward hacking. We highlight growing concerns about safety and governance, emphasizing the need for better evaluation frameworks and regulatory oversight.