mlsanityData to 7 Oct 2026

Sparks of Artificial General Intelligence: Early experiments with GPT-4

arXiv 2303.12712 ↗First posted 22 Mar 2023cs.CL cs.AIcited by 2,579

Not on the map. Why?

Cited by 379 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition. The latest model developed by OpenAI, GPT-4, was trained using an unprecedented scale of compute and data. In this paper, we report on our investigation of an early version of GPT-4, when it was still in active development by OpenAI. We contend that (this early version of) GPT-4 is part of a new cohort of LLMs (along with ChatGPT and Google's PaLM for example) that exhibit more general intelligence than previous AI models. We discuss the rising capabilities and implications of these models. We demonstrate that, beyond its mastery of language, GPT-4 can solve novel and difficult tasks that span mathematics, coding, vision, medicine, law, psychology and more, without needing any special prompting. Moreover, in all of these tasks, GPT-4's performance is strikingly close to human-level performance, and often vastly surpasses prior models such as ChatGPT. Given the breadth and depth of GPT-4's capabilities, we believe that it could reasonably be viewed as an early (yet still incomplete) version of an artificial general intelligence (AGI) system. In our exploration of GPT-4, we put special emphasis on discovering its limitations, and we discuss the challenges ahead for advancing towards deeper and more comprehensive versions of AGI, including the possible need for pursuing a new paradigm that moves beyond next-word prediction. We conclude with reflections on societal influences of the recent technological leap and future research directions.

Key facts. Cited by 2,579 papers in the map (data through 7 Oct 2026). Builds on 34 of its 86 references that are in the map. First posted on arXiv 22 Mar 2023.

Builds on

34 of 86 references.

References · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  8. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  9. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  10. 2022Emergent Abilities of Large Language Modelscited by 2,483
  11. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  12. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
22 more
  1. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  2. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  3. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  4. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  5. 2022In-context Learning and Induction Headscited by 930
  6. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  7. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  8. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
  9. 2022Transformers Learn Shortcuts to Automatacited by 260
  10. 2019Identifying and Reducing Gender Bias in Word-Level Language Modelscited by 214
  11. 2022Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limitcited by 184
  12. 2019On the Measure of Intelligencecited by 129
  13. 2022Unveiling Transformers with LEGO: a synthetic reasoning taskcited by 105
  14. 2022Benchmarking Spatial Relationships in Text-to-Image Generationcited by 96
  15. 2022The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymizationcited by 94
  16. 2022Vision Transformers provably learn spatial structurecited by 92
  17. 2022Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programmingcited by 91
  18. 2022GPT Takes the Bar Examcited by 65
  19. 2022Structured, flexible, and robust: benchmarking and improving large language models towards more human-like behavior in out-of-distribution reasoning taskscited by 56
  20. 2022Learning threshold neurons via the "edge of stability"cited by 50
  21. 2022Fixing Model Bugs with Natural Language Patchescited by 32
  22. 2023Dissociating language and thought in large language modelscited by 27

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 826; 2024: 999; 2025: 530; 2026: 224.

2,579 papers.

Sort:

By area:

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2023Let's Verify Step by Stepcited by 3,598
  4. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  5. 2023A Survey of Large Language Modelscited by 2,537
  6. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  7. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  8. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  9. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  10. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  11. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  12. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
38 more
  1. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  2. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  3. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  4. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  5. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  6. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  7. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  8. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
  9. 2023ChatDev: Communicative Agents for Software Developmentcited by 893
  10. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  11. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  12. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
  13. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
  14. 2023Large Language Modelscited by 774
  15. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  16. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
  17. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  18. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 677
  19. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  20. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
  21. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  22. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
  23. 2023WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instructcited by 618
  24. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 609
  25. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 580
  26. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 574
  27. 2023Textbooks Are All You Needcited by 561
  28. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  29. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  30. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
  31. 2023Segment Everything Everywhere All at Oncecited by 546
  32. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  33. 2024RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrievalcited by 519
  34. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
  35. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 491
  36. 2023Reinforced Self-Training (ReST) for Language Modelingcited by 486
  37. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 425
  38. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415

50 most cited of 2,579 All 2,579 (JSON)