mlsanityData to 7 Oct 2026

PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization

arXiv 2310.16427 ↗First posted 25 Oct 2023cs.CLcited by 226

Not on the map. Why?

Cited by 63 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Highly effective, task-specific prompts are often heavily engineered by experts to integrate detailed instructions and domain insights based on a deep understanding of both instincts of large language models (LLMs) and the intricacies of the target task. However, automating the generation of such expert-level prompts remains elusive. Existing prompt optimization methods tend to overlook the depth of domain knowledge and struggle to efficiently explore the vast space of expert-level prompts. Addressing this, we present PromptAgent, an optimization method that autonomously crafts prompts equivalent in quality to those handcrafted by experts. At its core, PromptAgent views prompt optimization as a strategic planning problem and employs a principled planning algorithm, rooted in Monte Carlo tree search, to strategically navigate the expert-level prompt space. Inspired by human-like trial-and-error exploration, PromptAgent induces precise expert-level insights and in-depth instructions by reflecting on model errors and generating constructive error feedback. Such a novel framework allows the agent to iteratively examine intermediate prompts (states), refine them based on error feedbacks (actions), simulate future rewards, and search for high-reward paths leading to expert prompts. We apply PromptAgent to 12 tasks spanning three practical domains: BIG-Bench Hard (BBH), as well as domain-specific and general NLP tasks, showing it significantly outperforms strong Chain-of-Thought and recent prompt optimization baselines. Extensive analyses emphasize its capability to craft expert-level, detailed, and domain-insightful prompts with great efficiency and generalizability.

Key facts. Cited by 226 papers in the map (data through 7 Oct 2026). Builds on 32 of its 50 references that are in the map. First posted on arXiv 25 Oct 2023.

Builds on

32 of 50 references.

References · 32

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  4. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  6. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  7. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  8. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  9. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  10. 2019How Can We Know What Language Models Know?cited by 953
  11. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  12. 2023Large Language Models as Optimizerscited by 830
20 more
  1. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  2. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 521
  3. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 496
  4. 2023Augmented Language Models: a Surveycited by 398
  5. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 361
  6. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 329
  7. 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 279
  8. 2022Generating Sequences by Learning to Self-Correctcited by 271
  9. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddingscited by 238
  10. 2023Do Large Language Models Know What They Don't Know?cited by 238
  11. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 236
  12. 2023REFINER: Reasoning Feedback on Intermediate Representationscited by 203
  13. 2022GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Modelscited by 127
  14. 2023Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learningcited by 115
  15. 2023Reprompting: Automated Chain-of-Thought Prompt Inference Through Gibbs Samplingcited by 46
  16. 2023Did You Read the Instructions? Rethinking the Effectiveness of Task Definitions in Instruction Learningcited by 40
  17. 2021Coherence boosting: When your pretrained language model is not paying enough attentioncited by 38
  18. 2022GPS: Genetic Prompt Search for Efficient Few-shot Learningcited by 37
  19. 2022ThinkSum: Probabilistic reasoning over sets using large language modelscited by 23
  20. 2023GPT is becoming a Turing machine: Here are some ways to program itcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 64; 2025: 83; 2026: 72.

226 papers.

Sort:

By area:

  1. 2024ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolutioncited by 266
  2. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 155
  3. 2024NATURAL PLAN: Benchmarking LLMs on Natural Language Planningcited by 125
  4. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  5. 2025Multi-Agent Collaboration via Evolving Orchestrationcited by 102
  6. 2024WebPilot: A Versatile and Autonomous Multi-Agent System for Web Task Execution with Strategic Explorationcited by 100
  7. 2024Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimizationcited by 93
  8. 2024AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generationcited by 80
  9. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 78
  10. 2024LLM Reasoners: New Evaluation, Library, and Analysis of Step-by-Step Reasoning with Large Language Modelscited by 70
  11. 2024A Systematic Survey on Large Language Models for Algorithm Designcited by 65
  12. 2024Trace is the Next AutoDiff: Generative Optimization with Rich Feedback, Execution Traces, and LLMscited by 62
38 more
  1. 2023Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planningcited by 59
  2. 2023Experiential Co-Learning of Software-Developing Agentscited by 57
  3. 2023Prompt Engineering a Prompt Engineercited by 53
  4. 2024AutoManual: Constructing Instruction Manuals by LLM Agents via Interactive Environmental Learningcited by 52
  5. 2024Large Language Models Can Solve Real-World Planning Rigorously with Formal Verification Toolscited by 47
  6. 2024Efficient Prompting Methods for Large Language Models: A Surveycited by 46
  7. 2025A Survey of Large Language Model Agents for Question Answeringcited by 46
  8. 2024Are Large Language Models Good Prompt Optimizers?cited by 37
  9. 2025Self-Supervised Prompt Optimizationcited by 37
  10. 2024Knowledge Boundary of Large Language Models: A Surveycited by 36
  11. 2024Prompt Optimization with Human Feedbackcited by 35
  12. 2023Prompt Optimization via Adversarial In-Context Learningcited by 33
  13. 2024Teach Better or Show Smarter? On Instructions and Exemplars in Automatic Prompt Optimizationcited by 28
  14. 2024Universal Prompt Optimizer for Safe Text-to-Image Generationcited by 27
  15. 2025Explain Before You Answer: A Survey on Compositional Visual Reasoningcited by 26
  16. 2024Fairer Preferences Elicit Improved Human-Aligned Large Language Model Judgmentscited by 25
  17. 2025Text2World: Benchmarking Large Language Models for Symbolic World Model Generationcited by 25
  18. 2024Steering Large Language Models between Code Execution and Textual Reasoningcited by 24
  19. 2025Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunitiescited by 23
  20. 2024Understanding the Importance of Evolutionary Search in Automated Heuristic Design with Large Language Modelscited by 22
  21. 2025Learning on the Job: An Experience-Driven Self-Evolving Agent for Long-Horizon Taskscited by 22
  22. 2026ReCreate: Reasoning and Creating Domain Agents Driven by Experiencecited by 22
  23. 2023Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Modelscited by 20
  24. 2025Agents of Change: Self-Evolving LLM Agents for Strategic Planningcited by 20
  25. 2024MetaReflection: Learning Instructions for Language Agents using Past Reflectionscited by 19
  26. 2025PromptSculptor: Multi-Agent Based Text-to-Image Prompt Optimizationcited by 19
  27. 2024CriSPO: Multi-Aspect Critique-Suggestion-guided Automatic Prompt Optimization for Text Generationcited by 18
  28. 2025System Prompt Optimization with Meta-Learningcited by 18
  29. 2025Online Multi-LLM Selection via Contextual Bandits under Unstructured Context Evolutioncited by 18
  30. 2024DOTS: Learning to Reason Dynamically in LLMs via Optimal Reasoning Trajectories Searchcited by 17
  31. 2024MORL-Prompt: An Empirical Analysis of Multi-Objective Reinforcement Learning for Discrete Prompt Optimizationcited by 16
  32. 2024Iterative Experience Refinement of Software-Developing Agentscited by 16
  33. 2024Raccoon: Prompt Extraction Benchmark of LLM-Integrated Applicationscited by 16
  34. 2024StraGo: Harnessing Strategic Guidance for Prompt Optimizationcited by 16
  35. 2024RePrompt: Planning by Automatic Prompt Engineering for Large Language Models Agentscited by 15
  36. 2025MASTER: A Multi-Agent System with LLM Specialized MCTScited by 15
  37. 2025Beyond Prompt Content: Enhancing LLM Performance via Content-Format Integrated Prompt Optimizationcited by 15
  38. 2025PlanGEN: A Multi-Agent Framework for Generating Planning and Reasoning Trajectories for Complex Problem Solvingcited by 15

50 most cited of 226 All 226 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Tournament of Prompts: Evolving LLM Instructions Through Structured Debates and Elo Ratings
  2. 2026SePO: Self-Evolving Prompt Agent for System Prompt Optimization
  3. 2024PRompt Optimization in Multi-Step Tasks (PROMST): Integrating Human Feedback and Heuristic-based Sampling
  4. 2023StrategyLLM: Large Language Models as Strategy Generators, Executors, Optimizers, and Evaluators for Problem Solving
  5. 2026Agent-GWO: Collaborative Agents for Dynamic Prompt Optimization in Large Language Models
  6. 2026Naive Prompt Optimization: Rethinking the Need for Complex Prompt Search
  7. 2026Environment-Grounded Automated Prompt Optimization for LLM Game Agents
  8. 2026Automatically Evolving Prompt Guidelines for Task-Specific Optimization