mlsanityData to 7 Oct 2026

Emergent Abilities of Large Language Models

arXiv 2206.07682 ↗First posted 15 Jun 2022cs.CLcited by 2,483

Not on the map. Why?

Cited by 375 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Scaling up language models has been shown to predictably improve performance and sample efficiency on a wide range of downstream tasks. This paper instead discusses an unpredictable phenomenon that we refer to as emergent abilities of large language models. We consider an ability to be emergent if it is not present in smaller models but is present in larger models. Thus, emergent abilities cannot be predicted simply by extrapolating the performance of smaller models. The existence of such emergence implies that additional scaling could further expand the range of capabilities of language models.

Key facts. Cited by 2,483 papers in the map (data through 7 Oct 2026). Builds on 70 of its 107 references that are in the map. First posted on arXiv 15 Jun 2022.

Builds on

70 of 107 references.

References · 70

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  9. 2020Scaling Laws for Neural Language Modelscited by 6,411
  10. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  11. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  12. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
38 more
  1. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  3. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  4. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  5. 2016Pointer Sentinel Mixture Modelscited by 2,992
  6. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  8. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  9. 2020Extracting Training Data from Large Language Modelscited by 2,144
  10. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  11. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  12. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  13. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  14. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  15. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  16. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  17. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  18. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  19. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  20. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  21. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  22. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  23. 2022Red Teaming Language Models with Language Modelscited by 983
  24. 2021Ethical and social risks of harm from Language Modelscited by 957
  25. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  26. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  27. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  28. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  29. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  30. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  31. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  32. 2018Universal Transformerscited by 613
  33. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  34. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  35. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 479
  36. 2018WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representationscited by 370
  37. 2022Deduplicating Training Data Mitigates Privacy Risks in Language Modelscited by 309
  38. 2021Unsolved Problems in ML Safetycited by 291

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 101; 2023: 751; 2024: 734; 2025: 552; 2026: 345.

2,483 papers.

Sort:

By area:

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  4. 2023Qwen Technical Reportcited by 3,273
  5. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  6. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  7. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  8. 2023A Survey of Large Language Modelscited by 2,537
  9. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  10. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  11. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  12. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
38 more
  1. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  2. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  3. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  4. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  5. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  6. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  7. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  8. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  9. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  10. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  11. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  12. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  13. 2023BloombergGPT: A Large Language Model for Financecited by 923
  14. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  15. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  16. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  17. 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
  18. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 805
  19. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  20. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
  21. 2023Large Language Modelscited by 774
  22. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  23. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 755
  24. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
  25. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  26. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  27. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 690
  28. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  29. 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 679
  30. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 677
  31. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 672
  32. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 671
  33. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  34. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
  35. 2022Large Language Models Can Self-Improvecited by 656
  36. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 643
  37. 2023A Comprehensive Overview of Large Language Modelscited by 639
  38. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612

50 most cited of 2,483 All 2,483 (JSON)