When Do Prompt-Side Agent Playbooks Transfer? Accuracy, Cost, and Runtime Shift in Agent Deployment
Not on the map. Why?
Abstract (arXiv)
Prompt-side playbooks can improve tool-using language agents without retraining, but their portability beyond the source setting is unclear. We study frozen playbook transfer under a shared distill--validate--transfer protocol. On ALFWorld, transfer is beneficial under controlled greedy decoding and, in one near-budget-matched comparison, distilled guidance outperforms five fixed demonstrations. On TAU2-Bench, a prespecified aggregate contrast supports a modest average matched-domain advantage, but global Holm correction retains only one of 135 route-level effects; the remaining grid provides descriptive evidence of compatibility-sensitive heterogeneity. On XBench-DeepSearch, one artifact--runtime pairing preserves useful first-try heuristics while producing repeated queries, delayed stopping, and substantial cost inflation after a context-runtime shift. Across benchmarks, transferred and target-derived playbooks both require target-side validation of success, termination, protocol compatibility, and cost. Frozen transfer is therefore a conditional cold-start option, not a reuse-by-default strategy or a universally preferable alternative to target-side redistillation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 11 of its 20 references that are in the map. First posted on arXiv 6 Aug 2026.
Builds on
11 of 20 references.
References · 11
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
- 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
- 2025A-MEM: Agentic Memory for LLM Agentscited by 1,063
- 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 233
- 2026Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skillscited by 187
- 2025How Memory Management Impacts LLM Agents: An Empirical Study of Experience-Following Behaviorcited by 101
- 2025Contextual Experience Replay for Self-Improvement of Language Agentscited by 48
- 2026Portable Agent Memory: A Protocol for Cryptographically-Verified Memory Transfer Across Heterogeneous AI Agentscited by 4
- 2026Training Language Agents to Learn from Experiencecited by 1
Built on by
Not cited yet.