# SPEAR: Code-Augmented Agentic Prompt Optimization arXiv: https://arxiv.org/abs/2605.26275 · HTML: https://mlsanity.com/paper/2605.26275/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. - Builds on 29 of its 72 references that are in the map. - First posted on arXiv 25 May 2026. ## Placed in - LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/) ## Builds on (most cited first, 29 of 29) - [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021) - [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022) - [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022) - [Automatic Prompt Optimization with "Gradient Descent" and Beam Search](https://mlsanity.com/paper/2305.03495/) (2023) - [Executable Code Actions Elicit Better LLM Agents](https://mlsanity.com/paper/2402.01030/) (2024) - [Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution](https://mlsanity.com/paper/2309.16797/) (2023) - [LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression](https://mlsanity.com/paper/2310.06839/) (2023) - [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020) - [Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programs](https://mlsanity.com/paper/2406.11695/) (2024) - [PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization](https://mlsanity.com/paper/2310.16427/) (2023) - [Multi-Agent Design: Optimizing Agents with Better Prompts and Topologies](https://mlsanity.com/paper/2502.02533/) (2025) - [Trace is the Next AutoDiff: Generative Optimization with Rich Feedback, Execution Traces, and LLMs](https://mlsanity.com/paper/2406.16218/) (2024) - [EvoX: Meta-Evolution for Automated Discovery](https://mlsanity.com/paper/2602.23413/) (2026) - [SkillForge: Forging Domain-Specific, Self-Evolving Agent Skills in Cloud Technical Support](https://mlsanity.com/paper/2604.08618/) (2026) - [PromptWizard: Task-Aware Prompt Optimization Framework](https://mlsanity.com/paper/2405.18369/) (2024) - [VISTA: A Test-Time Self-Improving Video Generation Agent](https://mlsanity.com/paper/2510.15831/) (2025) - [Symbolic Prompt Program Search: A Structure-Aware Approach to Efficient Compile-Time Prompt Optimization](https://mlsanity.com/paper/2404.02319/) (2024) - [Fairer Preferences Elicit Improved Human-Aligned Large Language Model Judgments](https://mlsanity.com/paper/2406.11370/) (2024) - [MARS: Multi-Agent Adaptive Reasoning with Socratic Guidance for Automated Prompt Optimization](https://mlsanity.com/paper/2503.16874/) (2025) - [No Loss, No Gain: Gated Refinement and Adaptive Compression for Prompt Optimization](https://mlsanity.com/paper/2509.23387/) (2025) - [ProCut: LLM Prompt Compression via Attribution Estimation](https://mlsanity.com/paper/2508.02053/) (2025) - [HPSS: Heuristic Prompting Strategy Search for LLM Evaluators](https://mlsanity.com/paper/2502.13031/) (2025) - [Auto-Prompt Ensemble for LLM Judge](https://mlsanity.com/paper/2510.06538/) (2025) - [Generalizable Self-Evolving Memory for Automatic Prompt Optimization](https://mlsanity.com/paper/2603.21520/) (2026) - [P3: Prompts Promote Prompting](https://mlsanity.com/paper/2507.15675/) (2025) - [Reflection in the Dark: Exposing and Escaping the Black Box in Reflective Prompt Optimization](https://mlsanity.com/paper/2603.18388/) (2026) - [Build, Judge, Optimize: A Blueprint for Continuous Improvement of Multi-Agent Consumer Assistants](https://mlsanity.com/paper/2603.03565/) (2026) - [REVERE: Reflective Evolving Research Engineer](https://mlsanity.com/paper/2603.20667/) (2026) - [ContraPrompt: Contrastive Prompt Optimization via Dyadic Reasoning Trace Analysis](https://mlsanity.com/paper/2604.17937/) (2026) ## Built on by (most cited first, 1 of 1) - [VERSE: Verified Self-Evolving Optimizer for Agent Harnesses](https://mlsanity.com/paper/2610.02616/) (2026) ## Abstract (arXiv) Automatic prompt engineering (APE) rewrites prompts to improve downstream task performance, but existing APE loops treat the optimizer itself as a fixed pipeline. We port the code-as-action paradigm of CodeAct (Wang et al., 2024a) to APE and propose SPEAR (Sandboxed Prompt Engineer with Active Roll-back), a free-form agentic optimizer with four tools -- evaluate, python, setprompt, finish -- that decides autonomously how and when to use them. The distinctive tool is the Python sandbox: the optimizer writes and executes arbitrary Python on the current evaluation DataFrame, performing structural error analysis (confusion matrices, error clustering, per group metrics) the agent itself authors. Two guardrails turn the long-horizon agent into a monotone-improving optimizer: auto-rollback on metric regression, and an optional guard metric floor. We evaluate on three industrial LLM-as-judge suites (13 judge tasks across recruiter-intake, conversational-memory, and query-refinement systems) plus seven BBH tasks and GSM8K. SPEAR wins every industrial task on the primary metric (kappa 0.857 vs 0.359 on tool-selection; F1-macro 0.815 vs 0.763 on filter-relevance; kappa 0.254 vs 0.218 on the hardest extraction dimension). On BBH-7 SPEAR averages 0.938 accuracy vs GEPA 0.628 and TextGrad 0.484. Ablations show the Python tool is the largest single lever on complex judge tasks (Delta ≈ +0.79kappa on the 5-class tool-selection judge, Delta ≈ +0.35kappa on the hardest extraction dimension when removed); its irreplaceable contribution is class-pair confusion aggregation that a long-context LLM cannot extract reliably from the raw eval DataFrame.