A Primer on Large Language Models and their Limitations
Not on the map. Why?
Abstract (arXiv)
This paper provides a primer on Large Language Models (LLMs) and identifies their strengths, limitations, applications and research directions. It is intended to be useful to those in academia and industry who are interested in gaining an understanding of the key LLM concepts and technologies, and in utilising this knowledge in both day to day tasks and in more complex scenarios where this technology can enhance current practices and processes.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 36 of its 78 references that are in the map. First posted on arXiv 3 Dec 2024.
Builds on
36 of 78 references.
References · 36
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
24 more
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
- 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
- 2022Training Compute-Optimal Large Language Modelscited by 3,019
- 2023A Survey of Large Language Modelscited by 2,576
- 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
- 2020Reformer: The Efficient Transformercited by 1,668
- 2016Progressive Neural Networkscited by 1,361
- 2018Experience Replay for Continual Learningcited by 1,095
- 2023BloombergGPT: A Large Language Model for Financecited by 939
- 2023A Comprehensive Overview of Large Language Modelscited by 648
- 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 639
- 2023The Curse of Recursion: Training on Generated Data Makes Models Forgetcited by 356
- 2023A Cookbook of Self-Supervised Learningcited by 250
- 2023Evaluating Large Language Models in Theory of Mind Taskscited by 222
- 2023Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuningcited by 173
- 2022What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?cited by 150
- 2017What do we need to build explainable AI systems for the medical domain?cited by 129
- 2024Revisiting Catastrophic Forgetting in Large Language Model Tuningcited by 99
- 2024The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunitiescited by 64
- 2020ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generatorscited by 27
- 2024YODA: Teacher-Student Progressive Learning for Language Modelscited by 10
- 2024Exploration of Masked and Causal Language Modelling for Text Generationcited by 7
- 2024MoDE: Effective Multi-task Parameter Efficient Fine-Tuning with a Mixture of Dyadic Expertscited by 6
- 2024Strategic AI Governance: Insights from Leading Nationscited by 1
Built on by
Citing papers per year: 2025: 3.
3 papers.