mlsanityData to 7 Oct 2026

A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT

arXiv 2302.11382 ↗First posted 21 Feb 2023cs.SE cs.AIcited by 612

Not on the map. Why?

Cited by 69 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Prompt engineering is an increasingly important skill set needed to converse effectively with large language models (LLMs), such as ChatGPT. Prompts are instructions given to an LLM to enforce rules, automate processes, and ensure specific qualities (and quantities) of generated output. Prompts are also a form of programming that can customize the outputs and interactions with an LLM. This paper describes a catalog of prompt engineering techniques presented in pattern form that have been applied to solve common problems when conversing with LLMs. Prompt patterns are a knowledge transfer method analogous to software patterns since they provide reusable solutions to common problems faced in a particular context, i.e., output generation and interaction when working with LLMs. This paper provides the following contributions to research on prompt engineering that apply LLMs to automate software development tasks. First, it provides a framework for documenting patterns for structuring prompts to solve a range of problems so that they can be adapted to different domains. Second, it presents a catalog of patterns that have been applied successfully to improve the outputs of LLM conversations. Third, it explains how prompts can be built from multiple patterns and illustrates prompt patterns that benefit from combination with other prompt patterns.

Key facts. Cited by 612 papers in the map (data through 7 Oct 2026). Builds on 20 of its 54 references that are in the map. First posted on arXiv 21 Feb 2023.

Builds on

20 of 54 references.

References · 20

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  3. 2022Emergent Abilities of Large Language Modelscited by 2,483
  4. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  5. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  6. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  7. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 888
  8. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  9. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 417
  10. 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 364
  11. 2023Mathematical Capabilities of ChatGPTcited by 263
  12. 2021PTR: Prompt Tuning with Rules for Text Classificationcited by 238
8 more
  1. 2021Design Guidelines for Prompt Engineering Text-to-Image Generative Modelscited by 216
  2. 2022Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanationscited by 159
  3. 2022Ask Me Anything: A simple strategy for prompting language modelscited by 155
  4. 2022Is GitHub's Copilot as Bad as Humans at Introducing Vulnerabilities in Code?cited by 63
  5. 2023Chat2VIS: Generating Data Visualisations via Natural Language using ChatGPT, Codex and GPT-3 Large Language Modelscited by 58
  6. 2023Can ChatGPT Write a Good Boolean Query for Systematic Review Literature Search?cited by 54
  7. 2023Conversational Automated Program Repaircited by 41
  8. 2017Applying Software Patterns to Address Interoperability in Blockchain-based Healthcare Appscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 141; 2024: 233; 2025: 156; 2026: 82.

612 papers.

Sort:

By area:

  1. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  2. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  3. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
  4. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  5. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 389
  6. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  7. 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 266
  8. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 255
  9. 2025Training Language Models to Reason Efficientlycited by 242
  10. 2023Multimodal Large Language Models: A Surveycited by 193
  11. 2024Security and Privacy Challenges of Large Language Models: A Surveycited by 179
  12. 2024The Effect of Sampling Temperature on Problem Solving in Large Language Modelscited by 168
38 more
  1. 2023Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Modelscited by 152
  2. 2023DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4cited by 128
  3. 2023Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselvescited by 123
  4. 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 122
  5. 2026SoK: Agentic Skills -- Beyond Tool Use in LLM Agentscited by 120
  6. 2023Self-correcting LLM-controlled Diffusion Modelscited by 112
  7. 2024SciAgents: Automating scientific discovery through multi-agent intelligent graph reasoningcited by 90
  8. 2024From LLMs to LLM-based Agents for Software Engineering: A Survey of Current, Challenges and Futurecited by 85
  9. 2024ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partitioncited by 81
  10. 2023Eight Things to Know about Large Language Modelscited by 73
  11. 2023Unleashing the potential of prompt engineering for large language modelscited by 72
  12. 2023Table-GPT: Table-tuned GPT for Diverse Table Taskscited by 69
  13. 2023FinGPT: Instruction Tuning Benchmark for Open-Source Large Language Models in Financial Datasetscited by 68
  14. 2024Alice in Wonderland: Simple Tasks Showing Complete Reasoning Breakdown in State-Of-the-Art Large Language Modelscited by 67
  15. 2023ChatGPT Prompt Patterns for Improving Code Quality, Refactoring, Requirements Elicitation, and Software Designcited by 66
  16. 2023The Self-Perception and Political Biases of ChatGPTcited by 66
  17. 2024Exploring LLM Prompting Strategies for Joint Essay Scoring and Feedback Generationcited by 63
  18. 2024The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Modelscited by 62
  19. 2024SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Typescited by 58
  20. 2024Design Principles for Generative AI Applicationscited by 57
  21. 2023One Small Step for Generative AI, One Giant Leap for AGI: A Complete Survey on ChatGPT in AIGC Eracited by 55
  22. 2024Determinants of LLM-assisted Decision-Makingcited by 55
  23. 2023Do LLMs Possess a Personality? Making the MBTI Test an Amazing Evaluation for Large Language Modelscited by 54
  24. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 54
  25. 2023Prompt Engineering for Healthcare: Methodologies and Applicationscited by 52
  26. 2023Evaluating Large Language Models on a Highly-specialized Topic, Radiation Oncology Physicscited by 49
  27. 2024DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processingcited by 49
  28. 2023TELeR: A General Taxonomy of LLM Prompts for Benchmarking Complex Taskscited by 48
  29. 2023Principled Instructions Are All You Need for Questioning LLaMA-1/2, GPT-3.5/4cited by 48
  30. 2024Explainable Artificial Intelligence: A Survey of Needs, Techniques, Applications, and Future Directioncited by 46
  31. 2023"HOT" ChatGPT: The promise of ChatGPT in detecting and discriminating hateful, offensive, and toxic comments on social mediacited by 45
  32. 2024A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicinecited by 45
  33. 2024FLEUR: An Explainable Reference-Free Evaluation Metric for Image Captioning Using a Large Multimodal Modelcited by 44
  34. 2024Fantastic Semantics and Where to Find Them: Investigating Which Layers of Generative LLMs Reflect Lexical Semanticscited by 43
  35. 2023A New Era in Software Security: Towards Self-Healing Software via Large Language Models and Formal Verificationcited by 42
  36. 2023Large Language Models as Tax Attorneys: A Case Study in Legal Capabilities Emergencecited by 42
  37. 2023The Unequal Opportunities of Large Language Models: Revealing Demographic Bias through Job Recommendationscited by 41
  38. 2023Disinformation Detection: An Evolving Challenge in the Age of LLMscited by 41

50 most cited of 612 All 612 (JSON)