mlsanityData to 7 Oct 2026

GPT-4 Technical Report

arXiv 2303.08774 ↗First posted 15 Mar 2023cs.CL cs.AIcited by 18,132

Not on the map. Why?

Cited by 2,931 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We report the development of GPT-4, a large-scale, multimodal model which can accept image and text inputs and produce text outputs. While less capable than humans in many real-world scenarios, GPT-4 exhibits human-level performance on various professional and academic benchmarks, including passing a simulated bar exam with a score around the top 10% of test takers. GPT-4 is a Transformer-based model pre-trained to predict the next token in a document. The post-training alignment process results in improved performance on measures of factuality and adherence to desired behavior. A core component of this project was developing infrastructure and optimization methods that behave predictably across a wide range of scales. This allowed us to accurately predict some aspects of GPT-4's performance based on models trained with no more than 1/1,000th the compute of GPT-4.

Key facts. Cited by 18,132 papers in the map (data through 7 Oct 2026). Builds on 57 of its 85 references that are in the map. First posted on arXiv 15 Mar 2023.

Builds on

57 of 85 references.

References · 57

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  8. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  9. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  10. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  11. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  12. 2020Scaling Laws for Neural Language Modelscited by 6,411
38 more
  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  2. 2016Layer Normalizationcited by 5,579
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  8. 2017Deep reinforcement learning from human preferencescited by 3,846
  9. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  10. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  11. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  12. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  13. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  14. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  15. 2022Emergent Abilities of Large Language Modelscited by 2,483
  16. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  17. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  18. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  19. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  20. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  21. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  22. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  23. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  24. 2022Red Teaming Language Models with Language Modelscited by 983
  25. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
  26. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  27. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  28. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  29. 2022Large Language Models Can Self-Improvecited by 656
  30. 2018Universal Transformerscited by 613
  31. 2017Deep Learning Scaling is Predictable, Empiricallycited by 556
  32. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  33. 2020Scaling Laws for Autoregressive Generative Modelingcited by 510
  34. 2022ST-MoE: Designing Stable and Transferable Sparse Expert Modelscited by 391
  35. 2022CodeT: Code Generation with Generated Testscited by 369
  36. 2022Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfercited by 272
  37. 2019WinoGrande: An Adversarial Winograd Schema Challenge at Scalecited by 194
  38. 2020The Computational Limits of Deep Learningcited by 188

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1779; 2024: 6265; 2025: 6833; 2026: 3255.

18,132 papers.

Sort:

By area:

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2024Qwen2.5 Technical Reportcited by 4,713
  3. 2024GPT-4o System Cardcited by 4,467
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  6. 2023Let's Verify Step by Stepcited by 3,598
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  8. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  9. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  10. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  11. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  12. 2023A Survey of Large Language Modelscited by 2,537
38 more
  1. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  2. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  3. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  4. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  5. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  6. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  7. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  8. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  9. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  10. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  11. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
  12. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  13. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
  14. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  15. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  16. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  17. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  18. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  19. 2025VGGT: Visual Geometry Grounded Transformercited by 1,475
  20. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,461
  21. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  22. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  23. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,411
  24. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  25. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  26. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  27. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  28. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  29. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
  30. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  31. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  32. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  33. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  34. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  35. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  36. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  37. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  38. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075

50 most cited of 18,132 All 18,132 (JSON)