mlsanityData to 7 Oct 2026

An Empirical Categorization of Prompting Techniques for Large Language Models: A Practitioner's Guide

arXiv 2402.14837 ↗First posted 18 Feb 2024cs.CL cs.AI cs.HCcited by 7

Not on the map. Why?

Abstract (arXiv)

Due to rapid advancements in the development of Large Language Models (LLMs), programming these models with prompts has recently gained significant attention. However, the sheer number of available prompt engineering techniques creates an overwhelming landscape for practitioners looking to utilize these tools. For the most efficient and effective use of LLMs, it is important to compile a comprehensive list of prompting techniques and establish a standardized, interdisciplinary categorization framework. In this survey, we examine some of the most well-known prompting techniques from both academic and practical viewpoints and classify them into seven distinct categories. We present an overview of each category, aiming to clarify their unique contributions and showcase their practical applications in real-world examples in order to equip fellow practitioners with a structured framework for understanding and categorizing prompting techniques tailored to their specific domains. We believe that this approach will help simplify the complex landscape of prompt engineering and enable more effective utilization of LLMs in various applications. By providing practitioners with a systematic approach to prompt categorization, we aim to assist in navigating the intricacies of effective prompt design for conversational pre-trained LLMs and inspire new possibilities in their respective fields.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 66 of its 88 references that are in the map. First posted on arXiv 18 Feb 2024.

Builds on

66 of 88 references.

References · 66

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  8. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  11. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  12. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
38 more
  1. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  2. 2023Mistral 7Bcited by 3,065
  3. 2022Emergent Abilities of Large Language Modelscited by 2,483
  4. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  5. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  6. 2019Language Models as Knowledge Bases?cited by 1,922
  7. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,371
  8. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  9. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  10. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  11. 2023Teaching Large Language Models to Self-Debugcited by 963
  12. 2023BloombergGPT: A Large Language Model for Financecited by 923
  13. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  14. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  15. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
  16. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  17. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  18. 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 612
  19. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  20. 2021MetaICL: Learning to Learn In Contextcited by 450
  21. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
  22. 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 387
  23. 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 364
  24. 2022A Systematic Evaluation of Large Language Models of Codecited by 360
  25. 2021True Few-Shot Learning with Language Modelscited by 343
  26. 2023Faithful Chain-of-Thought Reasoningcited by 285
  27. 2023Multilingual Jailbreak Challenges in Large Language Modelscited by 222
  28. 2023REFINER: Reasoning Feedback on Intermediate Representationscited by 203
  29. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 191
  30. 2022Ask Me Anything: A simple strategy for prompting language modelscited by 155
  31. 2023Fundamental Limitations of Alignment in Large Language Modelscited by 152
  32. 2023Progressive-Hint Prompting Improves Reasoning in Large Language Modelscited by 147
  33. 2023Auditing large language models: a three-layered approachcited by 135
  34. 2023Multimodal Prompting with Missing Modalities for Visual Recognitioncited by 121
  35. 2023Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Datacited by 119
  36. 2023An LLM can Fool Itself: A Prompt-Based Adversarial Attackcited by 111
  37. 2023Knowledge-Driven CoT: Exploring Faithful Reasoning in LLMs for Knowledge-intensive Question Answeringcited by 97
  38. 2023Conversational Health Agents: A Personalized LLM-Powered Agent Frameworkcited by 91

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 3; 2026: 3.

7 papers.