# From Agent Failures to Text Policies: What Works and What Breaks

arXiv: https://arxiv.org/abs/2607.20668 · HTML: https://mlsanity.com/paper/2607.20668/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 17 references that are in the map.
- First posted on arXiv 22 Jul 2026.

## Builds on (most cited first, 10 of 10)

- [ExpeL: LLM Agents Are Experiential Learners](https://mlsanity.com/paper/2308.10144/) (2023)
- [Automatic Prompt Optimization with "Gradient Descent" and Beam Search](https://mlsanity.com/paper/2305.03495/) (2023)
- [RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning](https://mlsanity.com/paper/2504.20073/) (2025)
- [TextWorld: A Learning Environment for Text-based Games](https://mlsanity.com/paper/1806.11532/) (2018)
- [Hindsight Credit Assignment for Long-Horizon LLM Agents](https://mlsanity.com/paper/2603.08754/) (2026)
- [Feedback Descent: Open-Ended Text Optimization via Pairwise Comparison](https://mlsanity.com/paper/2511.07919/) (2025)
- [TextWorldExpress: Simulating Text Games at One Million Steps Per Second](https://mlsanity.com/paper/2208.01174/) (2022)
- [Composing Policy Gradients and Prompt Optimization for Language Model Programs](https://mlsanity.com/paper/2508.04660/) (2025)
- [Kintsugi: Learning Policies by Repairing Executable Knowledge Bases](https://mlsanity.com/paper/2605.09487/) (2026)
- [Environment-Grounded Automated Prompt Optimization for LLM Game Agents](https://mlsanity.com/paper/2606.17838/) (2026)

## Abstract (arXiv)

TextGrad improves language-model systems by revising text from feedback. Its core thesis is that natural-language feedback can act as a gradient for optimizing text components without changing model weights. Applying it to agents is harder because feedback arrives only after a sequence of actions, making it difficult to identify which decision caused failure. We study this problem by separating the ability to follow a useful policy from the ability to learn that policy from experience. Our main finding is a clear gap between these two abilities. Human-written policies improve two frozen 7B agents on TextWorldExpress by 5.0 success points, showing that useful policy text exists. However, policies generated from agent trajectories do not reliably outperform fixed prompting, even with richer traces, counterfactual evidence, or iterative GEPA search. The main challenge for agent-level TextGrad is therefore not executing textual policy updates, but reliably generating and selecting them from experience.