# Improving Coherence and Persistence in Agentic AI for System Optimization

arXiv: https://arxiv.org/abs/2603.21321 · HTML: https://mlsanity.com/paper/2603.21321/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 22 of its 53 references that are in the map.
- First posted on arXiv 22 Mar 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 22 of 22)

- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [ExpeL: LLM Agents Are Experiential Learners](https://mlsanity.com/paper/2308.10144/) (2023)
- [GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning](https://mlsanity.com/paper/2507.19457/) (2025)
- [Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve](https://mlsanity.com/paper/2403.02310/) (2024)
- [ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution](https://mlsanity.com/paper/2509.19349/) (2025)
- [Context Length Alone Hurts LLM Performance Despite Perfect Retrieval](https://mlsanity.com/paper/2510.05381/) (2025)
- [Vidur: A Large-Scale Simulation Framework For LLM Inference](https://mlsanity.com/paper/2405.05465/) (2024)
- [An AI system to help scientists write expert-level empirical software](https://mlsanity.com/paper/2509.06503/) (2025)
- [Multi-objective Evolution of Heuristic Using Large Language Model](https://mlsanity.com/paper/2409.16867/) (2024)
- [Astra: A Multi-Agent System for GPU Kernel Performance Optimization](https://mlsanity.com/paper/2509.07506/) (2025)
- [SR-Scientist: Scientific Equation Discovery With Agentic AI](https://mlsanity.com/paper/2510.11661/) (2025)
- [CALM: Co-evolution of Algorithms and Language Model for Automatic Heuristic Design](https://mlsanity.com/paper/2505.12285/) (2025)
- [GPU Kernel Scientist: An LLM-Driven Framework for Iterative Kernel Optimization](https://mlsanity.com/paper/2506.20807/) (2025)
- [Scientific Algorithm Discovery by Augmenting AlphaEvolve with Deep Research](https://mlsanity.com/paper/2510.06056/) (2025)
- [Glia: A Human-Inspired AI for Automated Systems Design and Optimization](https://mlsanity.com/paper/2510.27176/) (2025)
- [How Far Are AI Scientists from Changing the World?](https://mlsanity.com/paper/2507.23276/) (2025)
- [Designing Network Algorithms via Large Language Models](https://mlsanity.com/paper/2404.01617/) (2024)
- [Improving Parallel Program Performance with LLM Optimizers via Agent-System Interfaces](https://mlsanity.com/paper/2410.15625/) (2024)
- [Fine-tuning Large Language Model for Automated Algorithm Design](https://mlsanity.com/paper/2507.10614/) (2025)
- [Man-Made Heuristics Are Dead. Long Live Code Generators!](https://mlsanity.com/paper/2510.08803/) (2025)
- [Congestion Control System Optimization with Large Language Models](https://mlsanity.com/paper/2508.16074/) (2025)
- [Towards Safer Heuristics With XPlain](https://mlsanity.com/paper/2410.15086/) (2024)

## Built on by (most cited first, 7 of 7)

- [VibeServe: Can AI Agents Build Bespoke LLM Serving Systems?](https://mlsanity.com/paper/2605.06068/) (2026)
- [Don't Gamble, GAMBLe: An Analytical Framework for AI-Driven Research Systems](https://mlsanity.com/paper/2606.02863/) (2026)
- [AIChilles: Automatically Uncovering Hidden Weaknesses in AI-Evolved Systems](https://mlsanity.com/paper/2606.15834/) (2026)
- [ArchAgent v2: A Case Study with the Data Prefetching Championship](https://mlsanity.com/paper/2608.09874/) (2026)
- [A Vocabulary for Multi-Agent Automated Research Systems](https://mlsanity.com/paper/2607.22682/) (2026)
- [ReForge: Keeping ABR Algorithms Never Finished with Verified Large Language Model Edits](https://mlsanity.com/paper/2608.15138/) (2026)
- [ROAR: Unifying Runs across Heterogeneous AI-Driven Research Systems](https://mlsanity.com/paper/2610.03966/) (2026)

## Abstract (arXiv)

Designing high-performance system heuristics is a creative, iterative process requiring experts to form hypotheses and execute multi-step conceptual shifts. While Large Language Models (LLMs) show promise in automating this loop, they struggle with complex system problems due to two critical failure modes: evolutionary neighborhood bias and the coherence ceiling. Evolutionary methods often remain trapped in local optima by relying on scalar benchmark scores, failing when coordinated multi-step changes are required. Conversely, existing agentic frameworks suffer from context degradation over long horizons or fail to accumulate knowledge across independent runs. We present Engram, an agentic researcher architecture that addresses these limitations by decoupling long-horizon exploration from the constraints of a single context window. Engram organizes exploration into a sequence of agents that iteratively design, test, and analyze mechanisms. At the conclusion of each run, an agent stores code snapshots, logs, and results in a persistent Archive and distills high-level modeling insights into a compact, persistent Research Digest. Subsequent agents then begin with a fresh context window, reading the Research Digest to build on prior discoveries. We find that Engram exhibits superior performance across diverse domains including multi-cloud multicast, LLM inference request routing, and optimizing KV cache reuse in databases with natural language queries.