# A Systematic Survey of Automatic Prompt Optimization Techniques

arXiv: https://arxiv.org/abs/2502.16923 · HTML: https://mlsanity.com/paper/2502.16923/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- Builds on 59 of its 103 references that are in the map.
- First posted on arXiv 24 Feb 2025.

## Builds on (most cited first, 30 of 59)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [ALFWorld: Aligning Text and Embodied Environments for Interactive Learning](https://mlsanity.com/paper/2010.03768/) (2020)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [Neural Network Acceptability Judgments](https://mlsanity.com/paper/1805.12471/) (2018)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [Adversarial NLI: A New Benchmark for Natural Language Understanding](https://mlsanity.com/paper/1910.14599/) (2019)
- [Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies](https://mlsanity.com/paper/2101.02235/) (2021)
- [Cross-Task Generalization via Natural Language Crowdsourcing Instructions](https://mlsanity.com/paper/2104.08773/) (2021)
- [Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problems](https://mlsanity.com/paper/1705.04146/) (2017)
- [Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution](https://mlsanity.com/paper/2309.16797/) (2023)
- [SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization](https://mlsanity.com/paper/1911.12237/) (2019)
- [RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning](https://mlsanity.com/paper/2205.12548/) (2022)
- ["Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection](https://mlsanity.com/paper/1705.00648/) (2017)
- [XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning](https://mlsanity.com/paper/2005.00333/) (2020)
- [ScienceWorld: Is your Agent Smarter than a 5th Grader?](https://mlsanity.com/paper/2203.07540/) (2022)
- [Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models](https://mlsanity.com/paper/2301.12661/) (2023)
- [The E2E Dataset: New Challenges For End-to-End Generation](https://mlsanity.com/paper/1706.09254/) (2017)

## Built on by (most cited first, 30 of 63)

- [A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence](https://mlsanity.com/paper/2507.21046/) (2025)
- [DeltaEvolve: Accelerating Scientific Discovery through Momentum-Driven Evolution](https://mlsanity.com/paper/2602.02919/) (2026)
- [EvoTool: Self-Evolving Tool-Use Policy Optimization in LLM Agents via Blame-Aware Mutation and Diversity-Aware Selection](https://mlsanity.com/paper/2603.04900/) (2026)
- [CooperBench: Why Coding Agents Cannot be Your Teammates Yet](https://mlsanity.com/paper/2601.13295/) (2026)
- [HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses](https://mlsanity.com/paper/2608.01918/) (2026)
- [Prompt-MII: Meta-Learning Instruction Induction for LLMs](https://mlsanity.com/paper/2510.16932/) (2025)
- [LLM Prompt Duel Optimizer: Efficient Label-Free Prompt Optimization](https://mlsanity.com/paper/2510.13907/) (2025)
- [Managing Procedural Memory in LLM Agents: Control, Adaptation, and Evaluation](https://mlsanity.com/paper/2606.23127/) (2026)
- [No Loss, No Gain: Gated Refinement and Adaptive Compression for Prompt Optimization](https://mlsanity.com/paper/2509.23387/) (2025)
- [Multimodal Prompt Optimization: Why Not Leverage Multiple Modalities for MLLMs](https://mlsanity.com/paper/2510.09201/) (2025)
- [Automatic Prompt Optimization for Knowledge Graph Construction: Insights from an Empirical Study](https://mlsanity.com/paper/2506.19773/) (2025)
- [ProCut: LLM Prompt Compression via Attribution Estimation](https://mlsanity.com/paper/2508.02053/) (2025)
- [GenPilot: A Multi-Agent System for Test-Time Prompt Optimization in Image Generation](https://mlsanity.com/paper/2510.07217/) (2025)
- [POPI: Personalizing LLMs via Optimized Natural Language Preference Inference](https://mlsanity.com/paper/2510.17881/) (2025)
- [COMMUNITYNOTES: A Dataset for Exploring the Helpfulness of Fact-Checking Explanations](https://mlsanity.com/paper/2510.24810/) (2025)
- [MEMO: Memory-Augmented Model Context Optimization for Robust Multi-Turn Multi-Agent LLM Games](https://mlsanity.com/paper/2603.09022/) (2026)
- [MAS-PromptBench: When Does Prompt Optimization Improve Multi-Agent LLM Systems?](https://mlsanity.com/paper/2606.23664/) (2026)
- [Beyond One-Size-Fits-All: Inversion Learning for Highly Effective NLG Evaluation Prompts](https://mlsanity.com/paper/2504.21117/) (2025)
- [Talk Less, Call Right: Enhancing Role-Play LLM Agents with Automatic Prompt Optimization and Role Prompting](https://mlsanity.com/paper/2509.00482/) (2025)
- [How to Auto-optimize Prompts for Domain Tasks? Adaptive Prompting and Reasoning through Evolutionary Domain Knowledge Adaptation](https://mlsanity.com/paper/2510.21148/) (2025)
- [Rubrics as an Attack Surface: Stealthy Preference Drift in LLM Judges](https://mlsanity.com/paper/2602.13576/) (2026)
- [Transduction is All You Need for Structured Data Workflows](https://mlsanity.com/paper/2508.15610/) (2025)
- [JudgeFlow: Agentic Workflow Optimization via Block Judge](https://mlsanity.com/paper/2601.07477/) (2026)
- [Prompt-Level Distillation: A Non-Parametric Alternative to Model Fine-Tuning for Efficient Reasoning](https://mlsanity.com/paper/2602.21103/) (2026)
- [Experience as a Compass: Multi-agent RAG with Evolving Orchestration and Agent Prompts](https://mlsanity.com/paper/2604.00901/) (2026)
- [AgentGrad: Intervention-guided Prompt Optimization for Multi Agent Systems](https://mlsanity.com/paper/2609.08572/) (2026)
- [Something Just Like TRuST : Toxicity Recognition of Span and Target](https://mlsanity.com/paper/2506.02326/) (2025)
- [Through the Judge's Eyes: Inferred Thinking Traces Improve Reliability of LLM Raters](https://mlsanity.com/paper/2510.25860/) (2025)
- [Textual Gradients are a Flawed Metaphor for Automatic Prompt Optimization](https://mlsanity.com/paper/2512.13598/) (2025)
- [MAC: Multi-Agent Constitution Learning](https://mlsanity.com/paper/2603.15968/) (2026)

## Abstract (arXiv)

Since the advent of large language models (LLMs), prompt engineering has been a crucial step for eliciting desired responses for various Natural Language Processing (NLP) tasks. However, prompt engineering remains an impediment for end users due to rapid advances in models, tasks, and associated best practices. To mitigate this, Automatic Prompt Optimization (APO) techniques have recently emerged that use various automated techniques to help improve the performance of LLMs on various tasks. In this paper, we present a comprehensive survey summarizing the current progress and remaining challenges in this field. We provide a formal definition of APO, a 5-part unifying framework, and then proceed to rigorously categorize all relevant works based on their salient features therein. We hope to spur further research guided by our framework.