Registry-Governed Agent Lifecycle:Completing EDDOps with Evaluation-DrivenRegistration, Promotion, and Retirement on AWS AgentCore
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
Enterprise adoption of LLM agents requires model selection methods that balance quality, reliability, safety, latency, and cost. Evaluation-Driven Development and Operations (EDDOps) positions evaluation as a continuous governing function across the agent lifecycle rather than a terminal checkpoint. This paper presents a practitioner-oriented instantiation of EDDOps on AWS Bedrock AgentCore and proposes a cost-to-performance framework for selecting foundation models in enterprise agent architectures. We make three contributions: a conceptual synthesis explaining why traditional TDD/BDD methods are insufficient for non-deterministic LLM agents; an architectural mapping of the EDDOps reference architecture onto AgentCore Runtime, Evaluations, Agent Registry, and CloudWatch observability; and an empirical cost-to-performance decision framework validated through a proof-of-concept comparing three foundation models across two deployment paths. Using trace data from 30 single-turn invocations across six agents, 9 multi-turn evaluations, and registry-integrated governance, we show how evaluation evidence can convert model selection from a benchmark-ranking exercise into a governed economic decision. The results suggest that managed agent platforms can support EDDOps when they provide trace-native observability, pluggable evaluator frameworks, and governed registry-based discovery.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 4 of its 16 references that are in the map. First posted on arXiv 1 Jul 2026.
Builds on
4 of 16 references.
Closest ancestors · 2
Same direction, newest first
- 2024Evaluation-Driven Development and Operations of LLM Agents: A Process Model and Reference Architecture21 Nov
- 2023AgentBench: Evaluating LLMs as Agents7 Aug
Other references · 2
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2026A Comparative Evaluation of AI Agent Security Guardrailscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Continuous Benchmark Generation for Evaluating Enterprise-scale LLM Agents
- 2025Benchmarking and Studying the LLM-based Agent System in End-to-End Software Development
- 2026BC-Bench: Evaluating Agentic Engineering in a Domain-Specific Language for ERP
- 2026Toward Architecture-Aware Evaluation Metrics for LLM Agents
- 2026Beyond Static Leaderboards: Predictive Validity for the Evaluation of LLM Agents
- 2026ERPBench: A State-Grounded Evaluation Paradigm for Computer-Use Agents in Enterprise Software
- 2026AgentWare: Automating the Lifecycle of Agentic Applications across the Edge-to-Cloud Continuum
- 2026General Agent Evaluation