mlsanityData to 7 Oct 2026

An Empirical Cost Attribution of Context-Compression Gateways in Multi-Turn Coding Agents

arXiv 2609.22114 ↗First posted 19 Aug 2026cs.CL cs.PF cs.SEcited by 0

On the map: LLM agents and tool use → Agent memory systems

also in 1 more
Abstract (arXiv)

Context compression is widely proposed as a way to cut the token bill of LLM coding agents, and public benchmarks report that aggressive compression preserves task-solving quality. These two facts do not imply the third one commonly assumed: that compressing file reads saves money in a real multi-turn agent. We instrument a production compression gateway (Paritok) between coding agents (Claude Code, Codex) and frontier LLMs (Claude Sonnet, GPT-5), and decompose the token bill of real sessions into three independent levers: tool-schema filtering, content compression of file reads and tool output, and history summarization. Measured in isolation under controlled A/B runs, the three save at fundamentally different rates. Tool-schema filtering removes a fixed block every turn, roughly 21K-57K tokens on a typical turn; it is linear in the turn count N and the only unambiguously and reproducibly positive lever. Content compression saves only about 2% of the cache-priced prefix per turn, but compressed reads accumulate in history and are re-sent on every later turn, so its cumulative saving grows quadratically, about 3350*N^2 tokens (measured), overtaking the fixed tool-filter saving within roughly 6 turns until the context window caps it. A non-destructive gateway lets the agent pull original bytes back on demand; each recall re-sends exactly the one segment just compressed away, so its cost is fixed and bounded rather than a multiplicative blowup, and heavy recall spends the accumulated saving back one segment at a time. Finally, a strong single-shot compression benchmark - 86.5% of SWE-bench quality retained at a 25.7% compression rate, achieved by the model this gateway deploys (Paritok-4B, reported separately) - is orthogonal to multi-turn agent cost and must not be cited as a cost-saving argument. We distill the results into an actionable recipe for where token-saving effort pays off.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 8 of its 14 references that are in the map. First posted on arXiv 19 Aug 2026.

Builds on

8 of 14 references.

References · 8

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  4. 2023Learning to Compress Prompts with Gist Tokenscited by 379
  5. 2023Adapting Language Models to Compress Contextscited by 350
  6. 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compressioncited by 343
  7. 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 279
  8. 2023Compressing Context to Enhance Inference Efficiency of Large Language Modelscited by 185

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Token Reduction Is Not Cost Reduction
  2. 2026Paritok-4B: Intent-Conditioned Context Compression for Coding Agents
  3. 2026Beyond Token Savings: A Systematic Study of Context Compression in LLM Agents
  4. 2025More with Less: An Empirical Study of Turn-Control Strategies for Efficient Coding Agents
  5. 2026Prompt Compression in Production Task Orchestration: A Pre-Registered Randomized Trial
  6. 2026Control Under Compression: Reliability Frontiers for Tool-Using Agents
  7. 2026CliffCompaction: Cost-Efficient Compaction for Long-Horizon Coding Agents
  8. 2026What Does Context Compression Cost an Agent? Interaction Costs Unrevealed by Task-Completion Metrics