# REVOLVE: Optimizing AI Systems by Tracking Response Evolution in Textual Optimization

arXiv: https://arxiv.org/abs/2412.03092 · HTML: https://mlsanity.com/paper/2412.03092/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 32 references that are in the map.
- First posted on arXiv 4 Dec 2024.

## Builds on (most cited first, 26 of 26)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [Automatic Prompt Optimization with "Gradient Descent" and Beam Search](https://mlsanity.com/paper/2305.03495/) (2023)
- [Learning to (Learn at Test Time): RNNs with Expressive Hidden States](https://mlsanity.com/paper/2407.04620/) (2024)
- [Faithful Chain-of-Thought Reasoning](https://mlsanity.com/paper/2301.13379/) (2023)
- [TextGrad: Automatic "Differentiation" via Text](https://mlsanity.com/paper/2406.07496/) (2024)
- [GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Models](https://mlsanity.com/paper/2203.07281/) (2022)
- [Symbolic Learning Enables Self-Evolving Agents](https://mlsanity.com/paper/2406.18532/) (2024)
- [Automatically Neutralizing Subjective Bias in Text](https://mlsanity.com/paper/1911.09709/) (2019)
- [Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimization](https://mlsanity.com/paper/2402.17574/) (2024)
- [Language Agents as Optimizable Graphs](https://mlsanity.com/paper/2402.16823/) (2024)
- [GUARD: Role-playing to Generate Natural-language Jailbreakings to Test Guideline Adherence of Large Language Models](https://mlsanity.com/paper/2402.03299/) (2024)
- [Offline Training of Language Model Agents with Functions as Learnable Weights](https://mlsanity.com/paper/2402.11359/) (2024)

## Built on by (most cited first, 20 of 20)

- [A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence](https://mlsanity.com/paper/2507.21046/) (2025)
- [Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systems](https://mlsanity.com/paper/2504.01990/) (2025)
- [FLEX: Continuous Agent Evolution via Forward Learning from Experience](https://mlsanity.com/paper/2511.06449/) (2025)
- [ReCreate: Reasoning and Creating Domain Agents Driven by Experience](https://mlsanity.com/paper/2601.11100/) (2026)
- [Beyond Magic Words: Sharpness-Aware Prompt Evolving for Robust Large Language Models with TARE](https://mlsanity.com/paper/2509.24130/) (2025)
- [GenoMAS: A Multi-Agent Framework for Scientific Discovery via Code-Driven Gene Expression Analysis](https://mlsanity.com/paper/2507.21035/) (2025)
- [SIPDO: Closed-Loop Prompt Optimization via Synthetic Data Feedback](https://mlsanity.com/paper/2505.19514/) (2025)
- [PREMISE: Scalable and Strategic Prompt Optimization for Efficient Mathematical Reasoning in Large Models](https://mlsanity.com/paper/2506.10716/) (2025)
- [Compound AI Systems Optimization: A Survey of Methods, Challenges, and Future Directions](https://mlsanity.com/paper/2506.08234/) (2025)
- [TextReg: Mitigating Prompt Distributional Overfitting via Regularized Text-Space Optimization](https://mlsanity.com/paper/2605.21318/) (2026)
- [DYNASHIELD: A Black-Box Moving Target Defense for LLMs via Dynamic Decoding Customization](https://mlsanity.com/paper/2412.07672/) (2024)
- [GuardVal: Dynamic Large Language Model Jailbreak Evaluation for Comprehensive Safety Testing](https://mlsanity.com/paper/2507.07735/) (2025)
- [Learning How and What to Memorize: Cognition-Inspired Two-Stage Optimization for Evolving Memory](https://mlsanity.com/paper/2605.00702/) (2026)
- [Are Prompt Optimizers Blind? Cross-Modal Visual Feedback for Automatic Prompt Optimization](https://mlsanity.com/paper/2607.24354/) (2026)
- [Overcoming the Weakest-Link Effect in LLM-Driven Program Optimization via Heterogeneous Edit Recombination](https://mlsanity.com/paper/2607.28947/) (2026)
- [Textual Equilibrium Propagation for Deep Compound AI Systems](https://mlsanity.com/paper/2601.21064/) (2026)
- [Synthetic Data-Driven Prompt Tuning for Financial QA over Tables and Documents](https://mlsanity.com/paper/2511.06292/) (2025)
- [TextBFGS: A Case-Based Reasoning Approach to Code Optimization via Error-Operator Retrieval](https://mlsanity.com/paper/2602.00059/) (2026)
- [STEVE: Stabilizing Textual Gradient-Based Prompt Optimization via Error-Driven Refinement and Regularized Verification](https://mlsanity.com/paper/2609.23716/) (2026)
- [Breaking Babel: A Self-Evolving Multi-Agent System for Long-Form Subtitle Translation](https://mlsanity.com/paper/2609.38660/) (2026)

## Abstract (arXiv)

Recent advancements in large language models (LLMs) have significantly enhanced the ability of LLM-based systems to perform complex tasks through natural language processing and tool interaction. However, optimizing these LLM-based systems for specific tasks remains challenging, often requiring manual interventions like prompt engineering and hyperparameter tuning. Existing automatic optimization methods, such as textual feedback-based techniques (e.g., TextGrad), tend to focus on immediate feedback, analogous to using immediate derivatives in traditional numerical gradient descent. However, relying solely on such feedback can be limited when the adjustments made in response to this feedback are either too small or fluctuate irregularly, potentially slowing down or even stalling the optimization process. To overcome these challenges, more adaptive methods are needed, especially in situations where the system's response is evolving slowly or unpredictably. In this paper, we introduce REVOLVE, an optimization method that tracks how "R"esponses "EVOLVE" across iterations in LLM systems. By focusing on the evolution of responses over time, REVOLVE enables more stable and effective optimization by making thoughtful, progressive adjustments at each step. Experimental results demonstrate that REVOLVE outperforms competitive baselines, achieving a 7.8% improvement in prompt optimization, a 20.72% gain in solution refinement, and a 29.17% increase in code optimization. Additionally, REVOLVE converges in fewer iterations, resulting in significant computational savings. Beyond its practical contributions, REVOLVE highlights a promising direction, where the rich knowledge from established optimization principles can be leveraged to enhance LLM systems, which paves the way for further advancements in this hybrid domain.