mlsanityData to 7 Oct 2026

Optimize Cheap, Deploy Strong: Cost-Aware Cross-Tier Transfer for Evolutionary Optimization

arXiv 2608.10694 ↗First posted 11 Aug 2026cs.LG cs.AI cs.CLcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Evolutionary optimization of LLM prompts and agentic programs (e.g., GEPA) is dominated by fitness evaluation: scoring each candidate runs an answering LLM over a validation set, so the evaluator's price tier dictates total search cost. We restructure that search by decoupling the three roles an LLM plays, running the high-volume answering role on the cheapest tier, reserving a strong model for the rare reflection/variation operator, then exploiting upward cross-tier transfer to deploy the cheaply evolved prompt on a stronger target. We contribute a cost-controlled characterization of when cheap-tier search substitutes for target-tier search, and where it fails. Across four tasks (HotpotQA, IFBench, LiveBench-Math, HoVer) and eleven models in four model families, the resulting prompt matches or exceeds same-tier optimization while placing over 96% of search tokens on the cheapest tier, at 5.6-14x lower search cost, rising to 25-54x where reasoning tiers emit long chains of thought on every fitness call.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 16 of its 31 references that are in the map. First posted on arXiv 11 Aug 2026.

Builds on

16 of 31 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026AdaptEvolve: Improving Efficiency of Evolutionary AI Agents through Adaptive Model Selection12 Feb

Other references · 15

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  2. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  3. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  4. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 688
  5. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
  6. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 521
  7. 2025GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningcited by 449
  8. 2024Optimizing Instructions and Demonstrations for Multi-Stage Language Model Programscited by 270
  9. 2025ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolutioncited by 146
  10. 2026LEVI: Stronger Search Architectures Can Substitute for Larger LLMs in Evolutionary Searchcited by 5
  11. 2026LABO: LLM-Accelerated Bayesian Optimization through Broad Exploration and Selective Experimentationcited by 4
  12. 2025Efficient and Privacy-Preserving Soft Prompt Transfer for LLMscited by 1
3 more
  1. 2026Efficient Hyperparameter Optimization for LLM Reinforcement Learningcited by 1
  2. 2026MAGE: Understanding Stability-Performance Trade-offs in Multi-component Prompt Optimizationcited by 1
  3. 2026Relay, Don't Route: Adaptive Population Handoff for Cost-Efficient LLM-Driven Evolutioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026What Makes an LLM a Good Optimizer? A Trajectory Analysis of LLM-Guided Evolutionary Search
  2. 2025Tournament of Prompts: Evolving LLM Instructions Through Structured Debates and Elo Ratings
  3. 2026Where Does Harness-Optimization Value Live? Localized Gains and the Budget-Splitting Trap in Self-Evolving LLM Agents
  4. 2026Evolution or Illusion? Rethinking Evaluation in LLM Evolutionary Search
  5. 2026FreeEvolve: Learning to Evolve Beyond Fixed Loops
  6. 2026PACEvolve++: Improving Test-time Learning for Evolutionary Search Agents
  7. 2026Compute Allocation for Self-Evolving LLMs: From Depth-Breadth to Multi-Armed Bandits
  8. 2025OPT-BENCH: Evaluating LLM Agent on Large-Scale Search Spaces Optimization Problems