# Parallel Context Compaction for Long-Horizon LLM Agent Serving

arXiv: https://arxiv.org/abs/2605.23296 · HTML: https://mlsanity.com/paper/2605.23296/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Agent memory systems, one of 16 directions in LLM agents and tool use.
- Builds on 25 of its 33 references that are in the map.
- First posted on arXiv 22 May 2026.

## Placed in

- LLM agents and tool use / [Agent memory systems](https://mlsanity.com/agents/agent-memory-systems/)

## Builds on (most cited first, 25 of 25)

- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://mlsanity.com/paper/2402.17753/) (2024)
- [LLM Evaluators Recognize and Favor Their Own Generations](https://mlsanity.com/paper/2404.13076/) (2024)
- [LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression](https://mlsanity.com/paper/2310.06839/) (2023)
- [Beyond Goldfish Memory: Long-Term Open-Domain Conversation](https://mlsanity.com/paper/2107.07567/) (2021)
- [Chain of Agents: Large Language Models Collaborating on Long-Context Tasks](https://mlsanity.com/paper/2406.02818/) (2024)
- [ACON: Optimizing Context Compression for Long-horizon LLM Agents](https://mlsanity.com/paper/2510.00615/) (2025)
- [Context Length Alone Hurts LLM Performance Despite Perfect Retrieval](https://mlsanity.com/paper/2510.05381/) (2025)
- [NoLiMa: Long-Context Evaluation Beyond Literal Matching](https://mlsanity.com/paper/2502.05167/) (2025)
- [Pride and Prejudice: LLM Amplifies Self-Bias in Self-Refinement](https://mlsanity.com/paper/2402.11436/) (2024)
- [Cartridges: Lightweight and general-purpose long context representations via self-study](https://mlsanity.com/paper/2506.06266/) (2025)
- [LongGenBench: Benchmarking Long-Form Generation in Long Context LLMs](https://mlsanity.com/paper/2409.02076/) (2024)
- [LIFBench: Evaluating the Instruction Following Performance and Stability of Large Language Models in Long-Context Scenarios](https://mlsanity.com/paper/2411.07037/) (2024)
- [Fast KV Compaction via Attention Matching](https://mlsanity.com/paper/2602.16284/) (2026)
- [On Context Utilization in Summarization with Large Language Models](https://mlsanity.com/paper/2310.10570/) (2023)
- [LLM×MapReduce: Simplified Long-Sequence Processing using Large Language Models](https://mlsanity.com/paper/2410.09342/) (2024)
- [LIFEBench: Evaluating Length Instruction Following in Large Language Models](https://mlsanity.com/paper/2505.16234/) (2025)
- [Context-Aware Hierarchical Merging for Long Document Summarization](https://mlsanity.com/paper/2502.00977/) (2025)
- [When Does Divide and Conquer Work for Long Context LLM? A Noise Decomposition Framework](https://mlsanity.com/paper/2506.16411/) (2025)
- [Long Context Scaling: Divide and Conquer via Multi-Agent Question-driven Collaboration](https://mlsanity.com/paper/2505.20625/) (2025)
- [Can LLMs Track Their Output Length? A Dynamic Feedback Mechanism for Precise Length Regulation](https://mlsanity.com/paper/2601.01768/) (2026)
- [Prompt-Based One-Shot Exact Length-Controlled Generation with LLMs](https://mlsanity.com/paper/2508.13805/) (2025)
- [Plan-and-Write: Structure-Guided Length Control for LLMs without Model Retraining](https://mlsanity.com/paper/2511.01807/) (2025)

## Built on by (most cited first, 5 of 5)

- [Governance Decay: How Context Compaction Silently Erases Safety Constraints in Long-Horizon LLM Agents](https://mlsanity.com/paper/2606.22528/) (2026)
- [KV-streams for Efficient Compaction in Agentic Reinforcement Learning](https://mlsanity.com/paper/2609.35750/) (2026)
- [Handover of In-Context Learning State Across Session Boundaries](https://mlsanity.com/paper/2608.14528/) (2026)
- [LAM: Efficient Lossy Agent Memory Framework With A Retrieval-Score Error Bound](https://mlsanity.com/paper/2609.32256/) (2026)
- [Glasseek: A Transparent Recipe for Training and Trustworthy Evaluation of DeepSearch Agents](https://mlsanity.com/paper/2610.16998/) (2026)

## Abstract (arXiv)

Long-horizon LLM agents accumulate growing conversation histories that eventually exceed the model's context window. Context compaction via LLM-based summarization keeps the conversation bounded, but summarization is inherently lossy and the blocking call stalls agent inference for tens of seconds. Moreover, the operator has no fine-grained control over summary volume since prompt instructions are largely ignored, and as context grows, both the amount of output tokens the model produces and the information it retains fluctuate substantially from run to run, making the agent's retained knowledge unpredictable across runs. We introduce parallel compaction for long-horizon agentic flows and characterize it against the sequential synchronous baseline across four backbones spanning 8B to 120B parameters, mixing dense and MoE architectures with reasoning and non-reasoning models, on the HotpotQA multi-hop QA and LoCoMo long-context dialogue benchmarks. Parallel compaction gives the operator fine-grained, predictable control over summary volume and enables more targeted prompt engineering per block. At matched compaction decode volume, it reduces end-to-end wall time and improves compaction throughput over the sequential baseline.