# Prompt Design and Engineering: Introduction and Advanced Methods

arXiv: https://arxiv.org/abs/2401.14423 · HTML: https://mlsanity.com/paper/2401.14423/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 20 references that are in the map.
- First posted on arXiv 24 Jan 2024.

## Builds on (most cited first, 13 of 13)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Gorilla: Large Language Model Connected with Massive APIs](https://mlsanity.com/paper/2305.15334/) (2023)
- [HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face](https://mlsanity.com/paper/2303.17580/) (2023)
- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://mlsanity.com/paper/2104.08786/) (2021)
- [SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models](https://mlsanity.com/paper/2303.08896/) (2023)
- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [Automatic Chain of Thought Prompting in Large Language Models](https://mlsanity.com/paper/2210.03493/) (2022)
- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models](https://mlsanity.com/paper/2305.18323/) (2023)
- [DERA: Enhancing Large Language Model Completions with Dialog-Enabled Resolving Agents](https://mlsanity.com/paper/2303.17071/) (2023)

## Built on by (most cited first, 30 of 45)

- [A Survey of Context Engineering for Large Language Models](https://mlsanity.com/paper/2507.13334/) (2025)
- [Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs](https://mlsanity.com/paper/2406.09136/) (2024)
- [Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model Optimizers](https://mlsanity.com/paper/2402.17564/) (2024)
- [Probing the limitations of multimodal language models for chemistry and materials research](https://mlsanity.com/paper/2411.16955/) (2024)
- [OnePiece: Bringing Context Engineering and Reasoning to Industrial Cascade Ranking System](https://mlsanity.com/paper/2509.18091/) (2025)
- [PersoBench: Benchmarking Personalized Response Generation in Large Language Models](https://mlsanity.com/paper/2410.03198/) (2024)
- [Large Language Models are Contrastive Reasoners](https://mlsanity.com/paper/2403.08211/) (2024)
- [Doubling Your Data in Minutes: Ultra-fast Tabular Data Generation via LLM-Induced Dependency Graphs](https://mlsanity.com/paper/2507.19334/) (2025)
- [Evaluating Mathematical Reasoning Across Large Language Models: A Fine-Grained Approach](https://mlsanity.com/paper/2503.10573/) (2025)
- [GPT-4.1 Sets the Standard in Automated Experiment Design Using Novel Python Libraries](https://mlsanity.com/paper/2508.00033/) (2025)
- [AutoSurvey2: Empowering Researchers with Next Level Automated Literature Surveys](https://mlsanity.com/paper/2510.26012/) (2025)
- [LAURA: Enhancing Code Review Generation with Context-Enriched Retrieval-Augmented LLM](https://mlsanity.com/paper/2512.01356/) (2025)
- [An Evaluation of Large Language Models on Text Summarization Tasks Using Prompt Engineering Techniques](https://mlsanity.com/paper/2507.05123/) (2025)
- [Beyond One-Way Influence: Bidirectional Opinion Dynamics in Multi-Turn Human-LLM Interactions](https://mlsanity.com/paper/2510.20039/) (2025)
- [Local Prompt Optimization](https://mlsanity.com/paper/2504.20355/) (2025)
- [Towards Goal-oriented Prompt Engineering for Large Language Models: A Survey](https://mlsanity.com/paper/2401.14043/) (2024)
- [The Future of Intelligent Healthcare: A Systematic Analysis and Discussion on the Integration and Impact of Robots Using Large Language Models for Healthcare](https://mlsanity.com/paper/2411.03287/) (2024)
- [Reflexive Prompt Engineering: A Framework for Responsible Prompt Engineering and Interaction Design](https://mlsanity.com/paper/2504.16204/) (2025)
- [RiOT: Efficient Prompt Refinement with Residual Optimization Tree](https://mlsanity.com/paper/2506.16389/) (2025)
- [LLM Based Bayesian Optimization for Prompt Search](https://mlsanity.com/paper/2510.04384/) (2025)
- [SoraNav: Adaptive UAV Task-Centric Navigation via Zeroshot VLM Reasoning](https://mlsanity.com/paper/2510.25191/) (2025)
- [Large language models replicate and predict human cooperation across experiments in game theory](https://mlsanity.com/paper/2511.04500/) (2025)
- [Reasoning Relay: Evaluating Stability and Interchangeability of Large Language Models in Mathematical Reasoning](https://mlsanity.com/paper/2512.20647/) (2025)
- [PCToolkit: A Unified Plug-and-Play Prompt Compression Toolkit of Large Language Models](https://mlsanity.com/paper/2403.17411/) (2024)
- [Open Artificial Knowledge](https://mlsanity.com/paper/2407.14371/) (2024)
- [An Empirically-grounded tool for Automatic Prompt Linting and Repair: A Case Study on Bias, Vulnerability, and Optimization in Developer Prompts](https://mlsanity.com/paper/2501.12521/) (2025)
- [Can Large Language Models Unveil the Mysteries? An Exploration of Their Ability to Unlock Information in Complex Scenarios](https://mlsanity.com/paper/2502.19973/) (2025)
- [Representing data in words: A context engineering approach](https://mlsanity.com/paper/2503.15509/) (2025)
- [Reliable Decision Support with LLMs: A Framework for Evaluating Consistency in Binary Text Classification Applications](https://mlsanity.com/paper/2505.14918/) (2025)
- [Ratas framework: A comprehensive genai-based approach to rubric-based marking of real-world textual exams](https://mlsanity.com/paper/2505.23818/) (2025)

## Abstract (arXiv)

Prompt design and engineering has rapidly become essential for maximizing the potential of large language models. In this paper, we introduce core concepts, advanced techniques like Chain-of-Thought and Reflection, and the principles behind building LLM-based agents. Finally, we provide a survey of tools for prompt engineers.