Textual Gradients are a Flawed Metaphor for Automatic Prompt Optimization
Not on the map. Why?
Abstract (arXiv)
A well-engineered prompt can increase the performance of large language models; automatic prompt optimization techniques aim to increase performance without requiring human effort to tune the prompts. One leading class of prompt optimization techniques introduces the analogy of textual gradients. We investigate the behavior of these textual gradient methods through a series of experiments and case studies. While such methods often result in a performance improvement, our experiments suggest that the gradient analogy does not accurately explain their behavior. Our insights may inform the selection of prompt optimization strategies, and development of new approaches.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 18 of its 51 references that are in the map. First posted on arXiv 15 Dec 2025.
Builds on
18 of 51 references.
References · 18
- 2022Generative Adversarial Networkscited by 5,603
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
- 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
- 2019How Can We Know What Language Models Know?cited by 953
- 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
- 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
- 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 521
- 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 294
- 2023EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizerscited by 189
- 2021Reframing Instructional Prompts to GPTk's Languagecited by 176
- 2023What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learningcited by 149
- 2023Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generationcited by 128
6 more
- 2024The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunitiescited by 63
- 2024Trace is the Next AutoDiff: Generative Optimization with Rich Feedback, Execution Traces, and LLMscited by 62
- 2025A Systematic Survey of Automatic Prompt Optimization Techniquescited by 61
- 2025A Self-Improving Coding Agentcited by 58
- 2023Prompt Optimization via Adversarial In-Context Learningcited by 33
- 2024MetaReflection: Learning Instructions for Language Agents using Past Reflectionscited by 19
Built on by
Citing papers per year: 2026: 2.
2 papers.