mlsanityData to 14 Oct 2026

Position: Key Claims in LLM Research Have a Long Tail of Footnotes

arXiv 2308.07120 ↗First posted 14 Aug 2023cs.CLcited by 20

Not on the map. Why?

Abstract (arXiv)

Much of the recent discourse within the ML community has been centered around Large Language Models (LLMs), their functionality and potential -- yet not only do we not have a working definition of LLMs, but much of this discourse relies on claims and assumptions that are worth re-examining. We contribute a definition of LLMs, critically examine five common claims regarding their properties (including 'emergent properties'), and conclude with suggestions for future research directions and their framing.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 91 of its 151 references that are in the map. First posted on arXiv 14 Aug 2023.

Builds on

91 of 151 references.

References · 91

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  7. 2020Scaling Laws for Neural Language Modelscited by 6,586
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  9. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  10. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  11. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  12. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
38 more
  1. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  3. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  4. 2022Emergent Abilities of Large Language Modelscited by 2,512
  5. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  6. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  7. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  8. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,414
  9. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  10. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
  11. 2019A Comprehensive Survey on Transfer Learningcited by 1,160
  12. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  13. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  14. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  15. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 876
  16. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 868
  17. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  18. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
  19. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 669
  20. 2022Large Language Models Can Self-Improvecited by 664
  21. 2023Are Emergent Abilities of Large Language Models a Mirage?cited by 523
  22. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
  23. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 439
  24. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 377
  25. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  26. 2023Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agentscited by 360
  27. 2021AI and the Everything in the Whole Wide World Benchmarkcited by 349
  28. 2021True Few-Shot Learning with Language Modelscited by 344
  29. 2022Data Distributional Properties Drive Emergent In-Context Learning in Transformerscited by 309
  30. 2013One Billion Word Benchmark for Measuring Progress in Statistical Language Modelingcited by 308
  31. 2020Pretrained Transformers Improve Out-of-Distribution Robustnesscited by 294
  32. 2021Scaling Laws for Transfercited by 272
  33. 2023How is ChatGPT's behavior changing over time?cited by 270
  34. 2023Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Taskscited by 246
  35. 2022Data Contamination: From Memorization to Exploitationcited by 236
  36. 2022Neural Theory-of-Mind? On the Limits of Social Intelligence in Large LMscited by 220
  37. 2023Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4cited by 202
  38. 2023GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Modelscited by 176

50 most cited of 91 All 91 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 10; 2026: 2.

20 papers.

Sort:

By area:

  1. 2024CroissantLLM: A Truly Bilingual French-English Language Modelcited by 53
  2. 2024Trust No Bot: Discovering Personal Disclosures in Human-LLM Conversations in the Wildcited by 47
  3. 2024Training on the Test Task Confounds Evaluation and Emergencecited by 34
  4. 2024Emergence in non-neural models: grokking modular arithmetic via average gradient outer productcited by 28
  5. 2024A Percolation Model of Emergence: Analyzing Transformers Trained on a Formal Languagecited by 26
  6. 2024Are Large Language Models Good Classifiers? A Study on Edit Intent Classification in Scientific Document Revisionscited by 11
  7. 2024"I Am the One and Only, Your Cyber BFF": Understanding the Impact of GenAI Requires Understanding the Impact of Anthropomorphic AIcited by 8
  8. 2025Large Language Models and Emergence: A Complex Systems Perspectivecited by 7
  9. 2025Generating Synthetic Data with Formal Privacy Guarantees: State of the Art and the Road Aheadcited by 4
  10. 2025Investigating Task Arithmetic for Zero-Shot Information Retrievalcited by 4
  11. 2025Giving AI Agents Access to Cryptocurrency and Smart Contracts Creates New Vectors of AI Harmcited by 3
  12. 2025When Large Language Models Know the Table: A Framework for Assessing Data Contamination in Tabular Datasetscited by 3
8 more
  1. 2025Limits to AI Growth: The Ecological and Social Consequences of Scalingcited by 2
  2. 2025Reassessing Active Learning Adoption in Contemporary NLP: A Community Surveycited by 2
  3. 2025Express Your Doubts -- Probabilistic World Modeling Should not be Based on Token logprobscited by 1
  4. 2026Strategic Polysemy in AI Discourse: A Philosophical Analysis of Language, Hype, and Powercited by 1
  5. 2024A quest through interconnected datasets: lessons from highly-cited ICASSP paperscited by 0
  6. 2025Local Differences, Global Lessons: Insights from Organisation Policies for International Legislationcited by 0
  7. 2025Neither Stochastic Parroting nor AGI: LLMs Solve Tasks through Context-Directed Extrapolation from Training Data Priorscited by 0
  8. 2026Training Witnesses: Trusting the Training without Trusting the Trainercited by 0