mlsanityData to 14 Oct 2026

A Comprehensive Overview of Large Language Models

arXiv 2307.06435 ↗First posted 12 Jul 2023cs.CLcited by 648

Not on the map. Why?

Cited by 100 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large Language Models (LLMs) have recently demonstrated remarkable capabilities in natural language processing tasks and beyond. This success of LLMs has led to a large influx of research contributions in this direction. These works encompass diverse topics such as architectural innovations, better training strategies, context length improvements, fine-tuning, multi-modal LLMs, robotics, datasets, benchmarking, efficiency, and more. With the rapid development of techniques and regular breakthroughs in LLM research, it has become considerably challenging to perceive the bigger picture of the advances in this direction. Considering the rapidly emerging plethora of literature on LLMs, it is imperative that the research community is able to benefit from a concise yet comprehensive overview of the recent developments in this field. This article provides an overview of the existing literature on a broad range of LLM-related concepts. Our self-contained comprehensive overview of LLMs discusses relevant background concepts along with covering the advanced topics at the frontier of research in LLMs. This review article is intended to not only provide a systematic survey but also a quick comprehensive reference for the researchers and practitioners to draw insights from extensive informative summaries of the existing works to advance the LLM research.

Key facts. Cited by 648 papers in the map (data through 14 Oct 2026). Builds on 368 of its 530 references that are in the map. First posted on arXiv 12 Jul 2023.

Builds on

368 of 530 references.

References · 368

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  6. 2022Training language models to follow instructions with human feedbackcited by 15,971
  7. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  8. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  9. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  10. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  11. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  12. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
38 more
  1. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2020Scaling Laws for Neural Language Modelscited by 6,586
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  7. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  9. 2016Layer Normalizationcited by 5,669
  10. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  11. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  12. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  13. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  14. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  15. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  16. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  17. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  18. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  19. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  20. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  21. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  22. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  23. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  24. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  25. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  26. 2021Program Synthesis with Large Language Modelscited by 3,637
  27. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  28. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  29. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  30. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  31. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  32. 2016Pointer Sentinel Mixture Modelscited by 3,096
  33. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  34. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  35. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  36. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  37. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  38. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955

50 most cited of 368 All 368 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 22; 2024: 180; 2025: 274; 2026: 172.

648 papers.

Sort:

By area:

  1. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
  2. 2025Small Language Models are the Future of Agentic AIcited by 270
  3. 2024Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Modelcited by 268
  4. 2023Can LLM-Generated Misinformation Be Detected?cited by 177
  5. 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstructioncited by 166
  6. 2024Large Language Model-Brained GUI Agents: A Surveycited by 162
  7. 2024A Survey on Large Language Model Acceleration based on KV Cache Managementcited by 131
  8. 2024Benchmarking LLMs via Uncertainty Quantificationcited by 126
  9. 2024AdaShield: Safeguarding Multimodal Large Language Models from Structure-based Attack via Adaptive Shield Promptingcited by 125
  10. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
  11. 2025Monte Carlo Tree Search for Comprehensive Exploration in LLM-Based Automatic Heuristic Designcited by 109
  12. 2024GUI Agents: A Surveycited by 103
38 more
  1. 2024A Survey on Multilingual Large Language Models: Corpora, Alignment, and Biascited by 84
  2. 2025Fin-R1: A Large Language Model for Financial Reasoning through Reinforcement Learningcited by 83
  3. 2024Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Modelscited by 76
  4. 2025Router-R1: Teaching LLMs Multi-Round Routing and Aggregation via Reinforcement Learningcited by 72
  5. 2024Visual-RolePlay: Universal Jailbreak Attack on MultiModal Large Language Models via Role-playing Image Charactercited by 68
  6. 2024Wisdom of the Silicon Crowd: LLM Ensemble Prediction Capabilities Rival Human Crowd Accuracycited by 67
  7. 2025MDocAgent: A Multi-Modal Multi-Agent Framework for Document Understandingcited by 67
  8. 2025SpatialCoT: Advancing Spatial Reasoning through Coordinate Alignment and Chain-of-Thought for Embodied Task Planningcited by 62
  9. 2024VoroNav: Voronoi-based Zero-shot Object Navigation with Large Language Modelcited by 57
  10. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 57
  11. 2024Fine Tuning vs. Retrieval Augmented Generation for Less Popular Knowledgecited by 50
  12. 2025UAVs Meet LLMs: Overviews and Perspectives Toward Agentic Low-Altitude Mobilitycited by 49
  13. 2025TLA: Tactile-Language-Action Model for Contact-Rich Manipulationcited by 47
  14. 2024Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignmentcited by 44
  15. 2025El Agente: An Autonomous Agent for Quantum Chemistrycited by 44
  16. 2025Text-to-CadQuery: A New Paradigm for CAD Generation with Scalable Large Model Capabilitiescited by 43
  17. 2024RAG-Fusion: a New Take on Retrieval-Augmented Generationcited by 42
  18. 2025MLLMs are Deeply Affected by Modality Biascited by 40
  19. 2024ShortcutsBench: A Large-Scale Real-world Benchmark for API-based Agentscited by 36
  20. 2025A Survey of Direct Preference Optimizationcited by 36
  21. 2024Unveiling Visual Perception in Language Models: An Attention Head Analysis Approachcited by 35
  22. 2024Large Language Models in Cybersecurity: State-of-the-Artcited by 34
  23. 2024Large Language Models for UAVs: Current State and Pathways to the Futurecited by 33
  24. 2024Contextual Feature Extraction Hierarchies Converge in Large Language Models and the Braincited by 32
  25. 2024LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scalecited by 31
  26. 2023PALM: Predicting Actions through Language Modelscited by 29
  27. 2024MatryoshkaKV: Adaptive KV Compression via Trainable Orthogonal Projectioncited by 29
  28. 2024SPRIG: Improving Large Language Model Performance by System Prompt Optimizationcited by 29
  29. 2023GRASP: A novel benchmark for evaluating language GRounding And Situated Physics understanding in multimodal language modelscited by 28
  30. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 28
  31. 2024Text Clustering with Large Language Model Embeddingscited by 27
  32. 2024Quantifying Feature Space Universality Across Large Language Models via Sparse Autoencoderscited by 27
  33. 2025Large Language Models for Multi-Robot Systems: A Surveycited by 27
  34. 2025We Should Identify and Mitigate Third-Party Safety Risks in MCP-Powered Agent Systemscited by 27
  35. 2024Modality-Fair Preference Optimization for Trustworthy MLLM Alignmentcited by 26
  36. 2025Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmapcited by 26
  37. 2025SpaceMind: Camera-Guided Modality Fusion for Spatial Reasoning in Vision-Language Modelscited by 26
  38. 2023Balancing Autonomy and Alignment: A Multi-Dimensional Taxonomy for Autonomous LLM-powered Multi-Agent Architecturescited by 25

50 most cited of 648 All 648 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024A critical review of methods and challenges in large language models
  2. 2024Survey of different Large Language Model Architectures: Trends, Benchmarks, and Challenges
  3. 2024MM-LLMs: Recent Advances in MultiModal Large Language Models
  4. 2025Domain Specific Benchmarks for Evaluating Multimodal Large Language Models
  5. 2025Speed Always Wins: A Survey on Efficient Architectures for Large Language Models
  6. 2024A Review of Multi-Modal Large Language and Vision Models
  7. 2024Exploring Advanced Large Language Models with LLMsuite
  8. 2025Self-Improvement in Multimodal Large Language Models: A Survey