mlsanityData to 7 Oct 2026

Large Language Models

arXiv 2307.05782 ↗First posted 11 Jul 2023cs.CL hep-th math.HOcited by 774

Not on the map. Why?

Cited by 82 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Artificial intelligence is making spectacular progress, and one of the best examples is the development of large language models (LLMs) such as OpenAI's GPT series. In these lectures, written for readers with a background in mathematics or physics, we give a brief history and survey of the state of the art, and describe the underlying transformer architecture in detail. We then explore some current ideas on how LLMs work and how models trained to predict the next word in a text are able to perform other tasks displaying intelligence.

Key facts. Cited by 774 papers in the map (data through 7 Oct 2026). Builds on 79 of its 178 references that are in the map. First posted on arXiv 11 Jul 2023.

Builds on

79 of 178 references.

References · 79

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  9. 2023Let's Verify Step by Stepcited by 3,598
  10. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  11. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  12. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
38 more
  1. 2022Emergent Abilities of Large Language Modelscited by 2,483
  2. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  3. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  4. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  5. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  6. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  7. 2016Group Equivariant Convolutional Networkscited by 1,154
  8. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  9. 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
  10. 2022In-context Learning and Induction Headscited by 930
  11. 2022Toy Models of Superpositioncited by 928
  12. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 919
  13. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 915
  14. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 831
  15. 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
  16. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  17. 2022What Can Transformers Learn In-Context? A Case Study of Simple Function Classescited by 700
  18. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 671
  19. 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 599
  20. 2019Benign Overfitting in Linear Regressioncited by 544
  21. 2022Beyond neural scaling laws: beating power law scaling via data pruningcited by 519
  22. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 515
  23. 2020Hopfield Networks is All You Needcited by 461
  24. 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 417
  25. 2021MiniF2F: a cross-system benchmark for formal Olympiad-level mathematicscited by 380
  26. 2021Explaining Neural Scaling Lawscited by 353
  27. 2014The Loss Surfaces of Multilayer Networkscited by 278
  28. 2022Draft, Sketch, and Prove: Guiding Formal Theorem Provers with Informal Proofscited by 278
  29. 2020Inductive Biases for Deep Learning of Higher-Level Cognitioncited by 273
  30. 2022Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfercited by 272
  31. 2021Scaling Laws for Transfercited by 265
  32. 2022Transformers Learn Shortcuts to Automatacited by 260
  33. 2020Neurosymbolic AI: The 3rd Wavecited by 253
  34. 2021Deep learning: a statistical viewpointcited by 239
  35. 2022The Parallelism Tradeoff: Limitations of Log-Precision Transformerscited by 191
  36. 2018To understand deep learning we need to understand kernel learningcited by 186
  37. 2022Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limitcited by 184
  38. 2023ProofNet: Autoformalizing and Formally Proving Undergraduate-Level Mathematicscited by 180

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 56; 2024: 249; 2025: 295; 2026: 174.

774 papers.

Sort:

By area:

  1. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  2. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  3. 2024A Survey on Large Language Models for Code Generationcited by 646
  4. 2024Long Context Transfer from Language to Visioncited by 502
  5. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
  6. 2023Detecting Language Model Attacks with Perplexitycited by 411
  7. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
  8. 2024AFlow: Automating Agentic Workflow Generationcited by 395
  9. 2023UltraFeedback: Boosting Language Models with Scaled AI Feedbackcited by 394
  10. 20243D-VLA: A 3D Vision-Language-Action Generative World Modelcited by 320
  11. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 319
  12. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
38 more
  1. 2024Humans or LLMs as the Judge? A Study on Judgement Biasescited by 314
  2. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  3. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  4. 2024Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Modelcited by 258
  5. 2023Magicoder: Empowering Code Generation with OSS-Instructcited by 251
  6. 2024Token-Budget-Aware LLM Reasoningcited by 242
  7. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  8. 2024From Persona to Personalization: A Survey on Role-Playing Language Agentscited by 206
  9. 2024ColPali: Efficient Document Retrieval with Vision Language Modelscited by 190
  10. 2023An LLM Compiler for Parallel Function Callingcited by 170
  11. 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 170
  12. 2024MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Makingcited by 162
  13. 2024Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillationcited by 159
  14. 2024Large Language Models(LLMs) on Tabular Data: Prediction, Generation, and Understanding -- A Surveycited by 157
  15. 2023Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answeringcited by 150
  16. 2023Benchmarking Cognitive Biases in Large Language Models as Evaluatorscited by 147
  17. 2024AdvPrompter: Fast Adaptive Adversarial Prompting for LLMscited by 146
  18. 2023Large Language Models Understand and Can be Enhanced by Emotional Stimulicited by 143
  19. 2024LAB-Bench: Measuring Capabilities of Language Models for Biology Researchcited by 138
  20. 2024Vulnerability Detection with Code Language Models: How Far Are We?cited by 133
  21. 2025SafeChain: Safety of Language Models with Long Chain-of-Thought Reasoning Capabilitiescited by 133
  22. 2023Large Language Models for Robotics: A Surveycited by 132
  23. 2024Tamper-Resistant Safeguards for Open-Weight LLMscited by 132
  24. 2025MemOS: A Memory OS for AI Systemcited by 129
  25. 2025Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memorycited by 129
  26. 2024Mobile Edge Intelligence for Large Language Models: A Contemporary Surveycited by 125
  27. 2023SmartPlay: A Benchmark for LLMs as Intelligent Agentscited by 123
  28. 2024AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Callscited by 121
  29. 2024Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQLcited by 118
  30. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generationcited by 118
  31. 2025Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agentscited by 112
  32. 2023Discuss Before Moving: Visual Language Navigation via Multi-expert Discussionscited by 108
  33. 2023A Survey of Text Watermarking in the Era of Large Language Modelscited by 102
  34. 2024Video-CCAM: Enhancing Video-Language Understanding with Causal Cross-Attention Masks for Short and Long Videoscited by 95
  35. 2024CRAG -- Comprehensive RAG Benchmarkcited by 94
  36. 2024Tug-of-War Between Knowledge: Exploring and Resolving Knowledge Conflicts in Retrieval-Augmented Language Modelscited by 93
  37. 2023Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineeringcited by 91
  38. 2024RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Modelcited by 91

50 most cited of 774 All 774 (JSON)