mlsanityData to 7 Oct 2026

Automatic Prompt Optimization with "Gradient Descent" and Beam Search

arXiv 2305.03495 ↗First posted 4 May 2023cs.CL cs.AI cs.LGcited by 628

Not on the map. Why?

Cited by 122 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large Language Models (LLMs) have shown impressive performance as general purpose agents, but their abilities remain highly dependent on prompts which are hand written with onerous trial-and-error effort. We propose a simple and nonparametric solution to this problem, Automatic Prompt Optimization (APO), which is inspired by numerical gradient descent to automatically improve prompts, assuming access to training data and an LLM API. The algorithm uses minibatches of data to form natural language "gradients" that criticize the current prompt. The gradients are then "propagated" into the prompt by editing the prompt in the opposite semantic direction of the gradient. These gradient descent steps are guided by a beam search and bandit selection procedure which significantly improves algorithmic efficiency. Preliminary results across three benchmark NLP tasks and the novel problem of LLM jailbreak detection suggest that Automatic Prompt Optimization can outperform prior prompt editing techniques and improve an initial prompt's performance by up to 31%, by using data to rewrite vague task descriptions into more precise annotation instructions.

Key facts. Cited by 628 papers in the map (data through 7 Oct 2026). Builds on 20 of its 44 references that are in the map. First posted on arXiv 4 May 2023.

Builds on

20 of 44 references.

References · 20

  1. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  2. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  3. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  4. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  5. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  6. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  7. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  8. 2023Teaching Large Language Models to Self-Debugcited by 963
  9. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
  10. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  11. 2021Learning How to Ask: Querying LMs with Mixtures of Soft Promptscited by 376
  12. 2017"Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detectioncited by 361
8 more
  1. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 361
  2. 2021WARP: Word-level Adversarial ReProgrammingcited by 220
  3. 2022Optimizing Prompts for Text-to-Image Generationcited by 204
  4. 2022Instruction Induction: From Few Examples to Natural Language Task Descriptionscited by 152
  5. 2022GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Modelscited by 127
  6. 2014Algorithms for multi-armed bandit problemscited by 63
  7. 2022TEMPERA: Test-Time Prompting via Reinforcement Learningcited by 50
  8. 2022GPS: Genetic Prompt Search for Efficient Few-shot Learningcited by 37

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 52; 2024: 149; 2025: 188; 2026: 239.

628 papers.

Sort:

By area:

  1. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  2. 2023Large Language Models as Optimizerscited by 830
  3. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  4. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
  5. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 556
  6. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 521
  7. 2024Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programscited by 270
  8. 2024ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolutioncited by 266
  9. 2024TextGrad: Automatic "Differentiation" via Textcited by 250
  10. 2026Meta-Harness: End-to-End Optimization of Model Harnessescited by 248
  11. 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 237
  12. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
38 more
  1. 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 226
  2. 2023Any-to-Any Generation via Composable Diffusioncited by 203
  3. 2023EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizerscited by 189
  4. 2024PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMscited by 187
  5. 2023A Simple LLM Framework for Long-Range Video Question-Answeringcited by 176
  6. 2023Talk like a Graph: Encoding Graphs for Large Language Modelscited by 175
  7. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 155
  8. 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attackscited by 150
  9. 2024AdvPrompter: Fast Adaptive Adversarial Prompting for LLMscited by 146
  10. 2023GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigationcited by 137
  11. 2023Large language models can accurately predict searcher preferencescited by 129
  12. 2023Rephrase and Respond: Let Large Language Models Ask Better Questions for Themselvescited by 123
  13. 2023Black-Box Prompt Optimization: Aligning Large Language Models without Model Trainingcited by 122
  14. 2024AgentScope: A Flexible yet Robust Multi-Agent Platformcited by 120
  15. 2025Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketchingcited by 116
  16. 2024FinCon: A Synthesized LLM Multi-Agent System with Conceptual Verbal Reinforcement for Enhanced Financial Decision Makingcited by 112
  17. 2025X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agentscited by 112
  18. 2023The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversationcited by 107
  19. 2025Multi-Agent Design: Optimizing Agents with Better Prompts and Topologiescited by 106
  20. 2023AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Modelscited by 102
  21. 2024Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimizationcited by 93
  22. 2024Improving Text-to-Image Consistency via Automatic Prompt Optimizationcited by 88
  23. 2024On Verbalized Confidence Scores for LLMscited by 83
  24. 2024Language Agents as Optimizable Graphscited by 81
  25. 2024Embodied LLM Agents Learn to Cooperate in Organized Teamscited by 81
  26. 2023Unleashing the potential of prompt engineering for large language modelscited by 72
  27. 2024Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?cited by 70
  28. 2026Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineeringcited by 68
  29. 2025WoW: Towards a World omniscient World model Through Embodied Interactioncited by 66
  30. 2024Retrieval Augmented Generation (RAG) and Beyond: A Comprehensive Survey on How to Make your LLMs use External Data More Wiselycited by 65
  31. 2024A Systematic Survey on Large Language Models for Algorithm Designcited by 65
  32. 2024Trace is the Next AutoDiff: Generative Optimization with Rich Feedback, Execution Traces, and LLMscited by 62
  33. 2023Prompt Engineering a Prompt Engineercited by 53
  34. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  35. 2024Efficient Prompting Methods for Large Language Models: A Surveycited by 46
  36. 2023Towards Reliable Misinformation Mitigation: Generalization, Uncertainty, and GPT-4cited by 45
  37. 2024MegaAgent: A Large-Scale Autonomous LLM-based Multi-Agent System Without Predefined SOPscited by 45
  38. 2024Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspectivecited by 44

50 most cited of 628 All 628 (JSON)