# PromptWizard: Task-Aware Prompt Optimization Framework

arXiv: https://arxiv.org/abs/2405.18369 · HTML: https://mlsanity.com/paper/2405.18369/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 103 references that are in the map.
- First posted on arXiv 28 May 2024.

## Builds on (most cited first, 19 of 19)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [Are NLP Models really able to Solve Simple Math Word Problems?](https://mlsanity.com/paper/2103.07191/) (2021)
- [Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problems](https://mlsanity.com/paper/1705.04146/) (2017)
- [Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models](https://mlsanity.com/paper/2305.04091/) (2023)
- [Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution](https://mlsanity.com/paper/2309.16797/) (2023)
- [RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning](https://mlsanity.com/paper/2205.12548/) (2022)
- [Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters](https://mlsanity.com/paper/2212.10001/) (2022)
- [Active Example Selection for In-Context Learning](https://mlsanity.com/paper/2211.04486/) (2022)
- [EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizers](https://mlsanity.com/paper/2309.08532/) (2023)
- [Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango](https://mlsanity.com/paper/2209.07686/) (2022)
- [Evaluating the Robustness of Neural Language Models to Input Perturbations](https://mlsanity.com/paper/2108.12237/) (2021)
- [On the Relation between Sensitivity and Accuracy in In-context Learning](https://mlsanity.com/paper/2209.07661/) (2022)
- [InstructZero: Efficient Instruction Optimization for Black-Box Large Language Models](https://mlsanity.com/paper/2306.03082/) (2023)
- [TEMPERA: Test-Time Prompting via Reinforcement Learning](https://mlsanity.com/paper/2211.11890/) (2022)
- [Prompt Optimization via Adversarial In-Context Learning](https://mlsanity.com/paper/2312.02614/) (2023)
- [Use Your INSTINCT: INSTruction optimization for LLMs usIng Neural bandits Coupled with Transformers](https://mlsanity.com/paper/2310.02905/) (2023)
- [How well do SOTA legal reasoning models support abductive reasoning?](https://mlsanity.com/paper/2304.06912/) (2023)

## Built on by (most cited first, 30 of 36)

- [Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents](https://mlsanity.com/paper/2605.30621/) (2026)
- [Adapting the Interface, Not the Model: Runtime Harness Adaptation for Deterministic LLM Agents](https://mlsanity.com/paper/2605.22166/) (2026)
- [Promptomatix: An Automatic Prompt Optimization Framework for Large Language Models](https://mlsanity.com/paper/2507.14241/) (2025)
- [Bandit-Based Prompt Design Strategy Selection Improves Prompt Optimizers](https://mlsanity.com/paper/2503.01163/) (2025)
- [CAPO: Cost-Aware Prompt Optimization](https://mlsanity.com/paper/2504.16005/) (2025)
- [Revisiting Prompt Optimization with Large Reasoning Models-A Case Study on Event Extraction](https://mlsanity.com/paper/2504.07357/) (2025)
- [ORFuzz: Fuzzing the "Other Side" of LLM Safety -- Testing Over-Refusal](https://mlsanity.com/paper/2508.11222/) (2025)
- [MAPGD: Multi-Agent Prompt Gradient Descent for Collaborative Prompt Optimization](https://mlsanity.com/paper/2509.11361/) (2025)
- [promptolution: A Unified, Modular Framework for Prompt Optimization](https://mlsanity.com/paper/2512.02840/) (2025)
- [Learning, Fast and Slow: Towards LLMs That Adapt Continually](https://mlsanity.com/paper/2605.12484/) (2026)
- [DIVE: Unlocking Self-Improvement in Frozen Language Models Through Diversity-Driven Skill Evolution](https://mlsanity.com/paper/2608.12486/) (2026)
- [Mind the Metrics: Patterns for Telemetry-Aware In-IDE AI Application Development using the Model Context Protocol (MCP)](https://mlsanity.com/paper/2506.11019/) (2025)
- [Do LLMs Adhere to Label Definitions? Examining Their Receptivity to External Label Definitions](https://mlsanity.com/paper/2509.02452/) (2025)
- [BOOST: Bootstrapping Strategy-Driven Reasoning Programs for Program-Guided Fact-Checking](https://mlsanity.com/paper/2504.02467/) (2025)
- [PMPO: Probabilistic Metric Prompt Optimization for Small and Large Language Models](https://mlsanity.com/paper/2505.16307/) (2025)
- [ACING: Actor-Critic for Instruction Learning in Black-Box LLMs](https://mlsanity.com/paper/2411.12736/) (2024)
- [Chain of Grounded Objectives: Bridging Process and Goal-oriented Prompting for Code Generation](https://mlsanity.com/paper/2501.13978/) (2025)
- [Grammar-Guided Evolutionary Search for Discrete Prompt Optimisation](https://mlsanity.com/paper/2507.10326/) (2025)
- [Prompt-with-Me: in-IDE Structured Prompt Management for LLM-Driven Software Engineering](https://mlsanity.com/paper/2509.17096/) (2025)
- [MA-SAPO: Multi-Agent Reasoning for Score-Aware Prompt Optimization](https://mlsanity.com/paper/2510.16635/) (2025)
- [Beyond Elicitation: Provision-based Prompt Optimization for Knowledge-Intensive Tasks](https://mlsanity.com/paper/2511.10465/) (2025)
- [Learning from Prompt itself: the Hierarchical Attribution Prompt Optimization](https://mlsanity.com/paper/2601.02683/) (2026)
- [PLR: Plackett-Luce for Reordering In-Context Learning Examples](https://mlsanity.com/paper/2603.21373/) (2026)
- [Contexting as Recommendation: Evolutionary Collaborative Filtering for Context Engineering](https://mlsanity.com/paper/2605.15721/) (2026)
- [MO-CAPO: Multi-Objective Cost-Aware Prompt Optimization](https://mlsanity.com/paper/2605.18869/) (2026)
- [SPEAR: Code-Augmented Agentic Prompt Optimization](https://mlsanity.com/paper/2605.26275/) (2026)
- [PromptPilot: Improving Human-AI Collaboration Through LLM-Enhanced Prompt Engineering](https://mlsanity.com/paper/2510.00555/) (2025)
- [ExecTune: Effective Steering of Black-Box LLMs with Guide Models](https://mlsanity.com/paper/2604.09741/) (2026)
- [Mind's Eye: A Benchmark of Visual Abstraction, Transformation and Composition for Multimodal LLMs](https://mlsanity.com/paper/2604.16054/) (2026)
- [Self-Prompting Small Language Models for Privacy-Sensitive Clinical Information Extraction](https://mlsanity.com/paper/2605.04221/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have transformed AI across diverse domains, with prompting being central to their success in guiding model outputs. However, manual prompt engineering is both labor-intensive and domain-specific, necessitating the need for automated solutions. We introduce PromptWizard, a novel, fully automated framework for discrete prompt optimization, utilizing a self-evolving, self-adapting mechanism. Through a feedback-driven critique and synthesis process, PromptWizard achieves an effective balance between exploration and exploitation, iteratively refining both prompt instructions and in-context examples to generate human-readable, task-specific prompts. This guided approach systematically improves prompt quality, resulting in superior performance across 45 tasks. PromptWizard excels even with limited training data, smaller LLMs, and various LLM architectures. Additionally, our cost analysis reveals a substantial reduction in API calls, token usage, and overall cost, demonstrating PromptWizard's efficiency, scalability, and advantages over existing prompt optimization strategies.