Total Cost of Agency: Exact Attribution of Memory Injection Cost in Multi-Agent LLM Workflows
On the map: LLM agents and tool use → Software engineering coding agents also fits: Tool learning and function calling
Abstract (arXiv)
Every node in a multi-agent large language model (LLM) workflow retrieves context from memory and injects it into its prompt, where those injected tokens are billed as input tokens at the same per-token price as the system prompt and the user query. Production observability tools report total token cost but do not separate the tokens a node generates from the tokens it is handed, so this component of the bill is invisible to the teams paying it. We introduce the Total Cost of Agency (TCA), a decomposition of multi-agent workflow cost into base prompt, inference, memory injection, miss penalty and context-accumulation components, and an exact attribution method: a two-pass, non-billable token count that measures injected tokens directly rather than estimating them from word-count proxies. On a 200-task enterprise benchmark executed against real model APIs, memory injection accounts for 13.6 percent of the variable cost a compile-time optimizer can act on, about 12 percent of the full billed cost, and its share rises from a structural zero at workflow depth one to 27.6 percent at depth six. Injected tokens grow linearly with depth over the measured range (R^2 = 0.9974, depths two through six); a quadratic fit yields a negative leading coefficient, so the data do not exhibit convex growth at these depths. We show the component is controllable at fixed model tier: reducing the retrieval window capacity from 32 to 2 entries lowers injected tokens by 28.7 percent with an accuracy change within seed-level variation. We report in full that our graph-rewriting transforms are approximately cost-neutral in isolation, that two of the five decomposition terms are zero by construction in this harness, and that total workflow cost is dominated by model tier assignment, which we hold fixed and treat as prior work. Prompt caching is not evaluated; all figures are for the uncached case.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 18 of its 25 references that are in the map. First posted on arXiv 20 Sep 2026.
Builds on
18 of 25 references.
Closest ancestors · 1
Same direction, newest first
Other references · 17
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
- 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
- 2023GAIA: a benchmark for General AI Assistantscited by 1,084
- 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
- 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
- 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
- 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
5 more
- 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 354
- 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compressioncited by 343
- 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 222
- 2023AutoMix: Automatically Mixing Language Modelscited by 116
- 2025MasRouter: Learning to Route LLMs for Multi-Agent Systemscited by 80
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026What Does a Token Cost? A Mixture-of-Agents Measurement of Sufficient Per-Token Compute
- 2026Cost-Aware Speculative Execution for LLM-Agent Workflows: An Integrated Five-Dimension Method
- 2026Are Online Skill and Memory Modules Always Worth Their Tokens? A Budget-Constrained Study of Web Agents
- 2026Token Reduction Is Not Cost Reduction
- 2026TokenCast: Forecasting Token Consumption During LLM Agent Execution
- 2026How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks
- 2026Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering
- 2026Beyond Token Savings: A Systematic Study of Context Compression in LLM Agents