mlsanityData to 7 Oct 2026

ChatGPT's One-year Anniversary: Are Open-Source Large Language Models Catching up?

arXiv 2311.16989 ↗First posted 28 Nov 2023cs.CLcited by 15

Not on the map. Why?

Abstract (arXiv)

Upon its release in late 2022, ChatGPT has brought a seismic shift in the entire landscape of AI, both in research and commerce. Through instruction-tuning a large language model (LLM) with supervised fine-tuning and reinforcement learning from human feedback, it showed that a model could answer human questions and follow instructions on a broad panel of tasks. Following this success, interests in LLMs have intensified, with new LLMs flourishing at frequent interval across academia and industry, including many start-ups focused on LLMs. While closed-source LLMs (e.g., OpenAI's GPT, Anthropic's Claude) generally outperform their open-source counterparts, the progress on the latter has been rapid with claims of achieving parity or even better on certain tasks. This has crucial implications not only on research but also on business. In this work, on the first anniversary of ChatGPT, we provide an exhaustive overview of this success, surveying all tasks where an open-source LLM has claimed to be on par or better than ChatGPT.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 156 of its 191 references that are in the map. First posted on arXiv 28 Nov 2023.

Builds on

156 of 191 references.

References · 156

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2017Proximal Policy Optimization Algorithmscited by 14,441
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  9. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  10. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  11. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  12. 2021Evaluating Large Language Models Trained on Codecited by 8,053
38 more
  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  7. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  8. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  10. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  11. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  12. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  13. 2021Program Synthesis with Large Language Modelscited by 3,509
  14. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  15. 2023Qwen Technical Reportcited by 3,273
  16. 2023Mistral 7Bcited by 3,065
  17. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  18. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  19. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  20. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  21. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  22. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  23. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  24. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  25. 2023A Survey of Large Language Modelscited by 2,537
  26. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  27. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  28. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  29. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  30. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  31. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  32. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  33. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  34. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  35. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  36. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  37. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  38. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259

50 most cited of 156 All 156 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 14.

15 papers.

Sort:

By area:

  1. 2024Financial Report Chunking for Effective Retrieval Augmented Generationcited by 46
  2. 2023Learning to Break: Knowledge-Enhanced Reasoning in Multi-Agent Debate Systemcited by 26
  3. 2024Can We Further Elicit Reasoning in LLMs? Critic-Guided Planning with Retrieval-Augmentation for Solving Challenging Taskscited by 22
  4. 2024TreeEval: Benchmark-Free Evaluation of Large Language Models through Tree Planningcited by 14
  5. 2024A Comprehensive Survey of Contamination Detection Methods in Large Language Modelscited by 12
  6. 2024Still Not Quite There! Evaluating Large Language Models for Comorbid Mental Health Diagnosiscited by 12
  7. 2024PPTC-R benchmark: Towards Evaluating the Robustness of Large Language Models for PowerPoint Task Completioncited by 9
  8. 2024Semantic Routing for Enhanced Performance of LLM-Assisted Intent-Based 5G Core Network Management and Orchestrationcited by 7
  9. 2024Metaheuristics and Large Language Models Join Forces: Toward an Integrated Optimization Approachcited by 6
  10. 2024Can Open-Source LLMs Compete with Commercial Models? Exploring the Few-Shot Performance of Current GPT Models in Biomedical Taskscited by 6
  11. 2024Narrative-of-Thought: Improving Temporal Reasoning of Large Language Models via Recounted Narrativescited by 5
  12. 2024Language Models as Efficient Reward Function Searchers for Custom-Environment Multi-Objective Reinforcementcited by 3
3 more
  1. 2024ParaICL: Towards Parallel In-Context Learningcited by 1
  2. 2024Large Language Models, and LLM-Based Agents, Should Be Used to Enhance the Digital Public Spherecited by 1
  3. 2024Explainable Search and Discovery of Visual Cultural Heritage Collections with Multimodal Large Language Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024ChatGPT Alternative Solutions: Large Language Models Survey
  2. 2023Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models
  3. 2025Challenges and Applications of Large Language Models: A Comparison of GPT and DeepSeek family of models
  4. 2023An Empirical Study of Instruction-tuning Large Language Models in Chinese
  5. 2023Can we trust the evaluation on ChatGPT?
  6. 2025A Showdown of ChatGPT vs DeepSeek in Solving Programming Tasks
  7. 2024Testing and Evaluation of Large Language Models: Correctness, Non-Toxicity, and Fairness
  8. 2023GPT-Fathom: Benchmarking Large Language Models to Decipher the Evolutionary Path towards GPT-4 and Beyond