mlsanityData to 14 Oct 2026

PromptWizard: Task-Aware Prompt Optimization Framework

arXiv 2405.18369 ↗First posted 28 May 2024cs.CL cs.AI cs.LGcited by 36

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have transformed AI across diverse domains, with prompting being central to their success in guiding model outputs. However, manual prompt engineering is both labor-intensive and domain-specific, necessitating the need for automated solutions. We introduce PromptWizard, a novel, fully automated framework for discrete prompt optimization, utilizing a self-evolving, self-adapting mechanism. Through a feedback-driven critique and synthesis process, PromptWizard achieves an effective balance between exploration and exploitation, iteratively refining both prompt instructions and in-context examples to generate human-readable, task-specific prompts. This guided approach systematically improves prompt quality, resulting in superior performance across 45 tasks. PromptWizard excels even with limited training data, smaller LLMs, and various LLM architectures. Additionally, our cost analysis reveals a substantial reduction in API calls, token usage, and overall cost, demonstrating PromptWizard's efficiency, scalability, and advantages over existing prompt optimization strategies.

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). Builds on 19 of its 103 references that are in the map. First posted on arXiv 28 May 2024.

Builds on

19 of 103 references.

References · 19

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  3. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  4. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
  5. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 650
  6. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 560
  7. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 551
  8. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 366
  9. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matterscited by 222
  10. 2022Active Example Selection for In-Context Learningcited by 196
  11. 2023EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizerscited by 193
  12. 2022Text and Patterns: For Effective Chain of Thought, It Takes Two to Tangocited by 128
7 more
  1. 2021Evaluating the Robustness of Neural Language Models to Input Perturbationscited by 87
  2. 2022On the Relation between Sensitivity and Accuracy in In-context Learningcited by 72
  3. 2023InstructZero: Efficient Instruction Optimization for Black-Box Large Language Modelscited by 68
  4. 2022TEMPERA: Test-Time Prompting via Reinforcement Learningcited by 50
  5. 2023Prompt Optimization via Adversarial In-Context Learningcited by 33
  6. 2023Use Your INSTINCT: INSTruction optimization for LLMs usIng Neural bandits Coupled with Transformerscited by 30
  7. 2023How well do SOTA legal reasoning models support abductive reasoning?cited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 17; 2026: 18.

36 papers.

Sort:

By area:

  1. 2026Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agentscited by 34
  2. 2026Adapting the Interface, Not the Model: Runtime Harness Adaptation for Deterministic LLM Agentscited by 27
  3. 2025Promptomatix: An Automatic Prompt Optimization Framework for Large Language Modelscited by 12
  4. 2025Bandit-Based Prompt Design Strategy Selection Improves Prompt Optimizerscited by 9
  5. 2025CAPO: Cost-Aware Prompt Optimizationcited by 7
  6. 2025Revisiting Prompt Optimization with Large Reasoning Models-A Case Study on Event Extractioncited by 6
  7. 2025ORFuzz: Fuzzing the "Other Side" of LLM Safety -- Testing Over-Refusalcited by 6
  8. 2025MAPGD: Multi-Agent Prompt Gradient Descent for Collaborative Prompt Optimizationcited by 6
  9. 2025promptolution: A Unified, Modular Framework for Prompt Optimizationcited by 6
  10. 2026Learning, Fast and Slow: Towards LLMs That Adapt Continuallycited by 6
  11. 2026DIVE: Unlocking Self-Improvement in Frozen Language Models Through Diversity-Driven Skill Evolutioncited by 5
  12. 2025Mind the Metrics: Patterns for Telemetry-Aware In-IDE AI Application Development using the Model Context Protocol (MCP)cited by 4
24 more
  1. 2025Do LLMs Adhere to Label Definitions? Examining Their Receptivity to External Label Definitionscited by 4
  2. 2025BOOST: Bootstrapping Strategy-Driven Reasoning Programs for Program-Guided Fact-Checkingcited by 3
  3. 2025PMPO: Probabilistic Metric Prompt Optimization for Small and Large Language Modelscited by 3
  4. 2024ACING: Actor-Critic for Instruction Learning in Black-Box LLMscited by 2
  5. 2025Chain of Grounded Objectives: Bridging Process and Goal-oriented Prompting for Code Generationcited by 1
  6. 2025Grammar-Guided Evolutionary Search for Discrete Prompt Optimisationcited by 1
  7. 2025Prompt-with-Me: in-IDE Structured Prompt Management for LLM-Driven Software Engineeringcited by 1
  8. 2025MA-SAPO: Multi-Agent Reasoning for Score-Aware Prompt Optimizationcited by 1
  9. 2025Beyond Elicitation: Provision-based Prompt Optimization for Knowledge-Intensive Taskscited by 1
  10. 2026Learning from Prompt itself: the Hierarchical Attribution Prompt Optimizationcited by 1
  11. 2026PLR: Plackett-Luce for Reordering In-Context Learning Examplescited by 1
  12. 2026Contexting as Recommendation: Evolutionary Collaborative Filtering for Context Engineeringcited by 1
  13. 2026MO-CAPO: Multi-Objective Cost-Aware Prompt Optimizationcited by 1
  14. 2026SPEAR: Code-Augmented Agentic Prompt Optimizationcited by 1
  15. 2025PromptPilot: Improving Human-AI Collaboration Through LLM-Enhanced Prompt Engineeringcited by 0
  16. 2026ExecTune: Effective Steering of Black-Box LLMs with Guide Modelscited by 0
  17. 2026Mind's Eye: A Benchmark of Visual Abstraction, Transformation and Composition for Multimodal LLMscited by 0
  18. 2026Self-Prompting Small Language Models for Privacy-Sensitive Clinical Information Extractioncited by 0
  19. 2026Learning to Translate from Soft to Hard LLM Promptscited by 0
  20. 2026Towards Spec Learning: Inference-Time Alignment from Preference Pairscited by 0
  21. 2026Mara Chain: Rethinking Failure as a Stepping Stone for AI System Auto-Evolutioncited by 0
  22. 2026DUET: Co-Evolving Solver and Grader Agentscited by 0
  23. 2026Reflective Prompt Optimization of a Deep-Research Agent: A Topic-Disjoint, Multi-Seed Protocol for Measuring Transferable Gains and Prompt-Level Overfittingcited by 0
  24. 2026SpecForge: Recursive Self-Improvement for Coding Agents via Behavioral Specificationscited by 0