mlsanityData to 14 Oct 2026

Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond

arXiv 2304.13712 ↗First posted 26 Apr 2023cs.CL cs.AI cs.LGcited by 411

Not on the map. Why?

Cited by 28 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper presents a comprehensive and practical guide for practitioners and end-users working with Large Language Models (LLMs) in their downstream natural language processing (NLP) tasks. We provide discussions and insights into the usage of LLMs from the perspectives of models, data, and downstream tasks. Firstly, we offer an introduction and brief summary of current GPT- and BERT-style LLMs. Then, we discuss the influence of pre-training data, training data, and test data. Most importantly, we provide a detailed discussion about the use and non-use cases of large language models for various natural language processing tasks, such as knowledge-intensive tasks, traditional natural language understanding tasks, natural language generation tasks, emergent abilities, and considerations for specific tasks.We present various use cases and non-use cases to illustrate the practical applications and limitations of LLMs in real-world scenarios. We also try to understand the importance of data and the specific challenges associated with each NLP task. Furthermore, we explore the impact of spurious biases on LLMs and delve into other essential considerations, such as efficiency, cost, and latency, to ensure a comprehensive understanding of deploying LLMs in practice. This comprehensive guide aims to provide researchers and practitioners with valuable insights and best practices for working with LLMs, thereby enabling the successful implementation of these models in a wide range of NLP tasks. A curated list of practical guide resources of LLMs, regularly updated, can be found at urlhttps://github.com/Mooler0410/LLMsPracticalGuide.

Key facts. Cited by 411 papers in the map (data through 14 Oct 2026). Builds on 114 of its 171 references that are in the map. First posted on arXiv 26 Apr 2023.

Builds on

114 of 171 references.

References · 114

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  7. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  8. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  9. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  10. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  11. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  12. 2020Scaling Laws for Neural Language Modelscited by 6,586
38 more
  1. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  7. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  9. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  10. 2021Program Synthesis with Large Language Modelscited by 3,637
  11. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  12. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  13. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  14. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  15. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  16. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  17. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  18. 2023A Survey of Large Language Modelscited by 2,576
  19. 2022Emergent Abilities of Large Language Modelscited by 2,512
  20. 2018Deep contextualized word representationscited by 2,290
  21. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  22. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  23. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  24. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  25. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  26. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  27. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  28. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  29. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  30. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  31. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  32. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  33. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
  34. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  35. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  36. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  37. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,102
  38. 2019Cross-lingual Language Model Pretrainingcited by 1,090

50 most cited of 114 All 114 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 89; 2024: 169; 2025: 112; 2026: 41.

411 papers.

Sort:

By area:

  1. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  2. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  3. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 668
  4. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 634
  5. 2023Instruction Tuning for Large Language Models: A Surveycited by 623
  6. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 399
  7. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 387
  8. 2023Explainability for Large Language Models: A Surveycited by 333
  9. 2023Sentiment Analysis in the Era of Large Language Models: A Reality Checkcited by 268
  10. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 229
  11. 2023Large Language Models in Finance: A Surveycited by 226
  12. 2024A Survey on Efficient Inference for Large Language Modelscited by 205
38 more
  1. 2023Efficient Large Language Models: A Surveycited by 186
  2. 2023Can LLM-Generated Misinformation Be Detected?cited by 177
  3. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 163
  4. 2023Knowledge Graph Prompting for Multi-Document Question Answeringcited by 152
  5. 2023SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialectscited by 152
  6. 2024LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuningcited by 136
  7. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 135
  8. 2024Benchmarking LLMs via Uncertainty Quantificationcited by 126
  9. 2023A Survey of Large Language Models in Medicine: Progress, Application, and Challengecited by 124
  10. 2024Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQLcited by 120
  11. 2023Opportunities and Challenges for ChatGPT and Large Language Models in Biomedicine and Healthcited by 118
  12. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 110
  13. 2024On-Device Language Models: A Comprehensive Reviewcited by 104
  14. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 103
  15. 2024Large Language Models Meet NLP: A Surveycited by 103
  16. 2023LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunitiescited by 97
  17. 2023MemoChat: Tuning LLMs to Use Memos for Consistent Long-Range Open-Domain Conversationcited by 97
  18. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 93
  19. 2024Evolutionary Computation in the Era of Large Language Model: Survey and Roadmapcited by 90
  20. 2024I-Design: Personalized LLM Interior Designercited by 87
  21. 2024From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Futurecited by 86
  22. 2023A Survey of Graph Meets Large Language Model: Progress and Future Directionscited by 85
  23. 2024Self-playing Adversarial Language Game Enhances LLM Reasoningcited by 83
  24. 2024Scientific Large Language Models: A Survey on Biological & Chemical Domainscited by 80
  25. 2024A Survey of Large Language Models in Finance (FinLLMs)cited by 79
  26. 2024Detecting Multimedia Generated by Large AI Models: A Surveycited by 76
  27. 2023Dense and Aligned Captions (DAC) Promote Compositional Reasoning in VL Modelscited by 72
  28. 2023Large Language Models Can be Lazy Learners: Analyze Shortcuts in In-Context Learningcited by 69
  29. 2024Large Language Models for Cyber Security: A Systematic Literature Reviewcited by 68
  30. 2024Learning to Route LLMs with Confidence Tokenscited by 67
  31. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 66
  32. 2024LLM Discussion: Enhancing the Creativity of Large Language Models via Discussion Framework and Role-Playcited by 66
  33. 2023FATE-LLM: A Industrial Grade Federated Learning Framework for Large Language Modelscited by 63
  34. 2024LaMP: Language-Motion Pretraining for Motion Generation, Retrieval, and Captioningcited by 59
  35. 2023CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translationcited by 57
  36. 2024A Comprehensive Survey of Foundation Models in Medicinecited by 56
  37. 2023TPTU: Large Language Model-based AI Agents for Task Planning and Tool Usagecited by 55
  38. 2023Text2KGBench: A Benchmark for Ontology-Driven Knowledge Graph Generation from Textcited by 48

50 most cited of 411 All 411 (JSON)