Compress the Context, Keep the Commitments: A Formal Framework for Verifiable LLM Context Compression
On the map: Efficient LLM inference and serving → Prompt and context compression
Abstract (arXiv)
LLM context is not just tokens; it is a set of commitments. Long-running conversations accumulate goals, constraints, decisions, preferences, tool results, retrieved evidence, artifacts, and safety boundaries that future responses must preserve. Existing context-management methods reduce length through truncation, retrieval, summarization, memory systems, or token-level prompt compression, but they rarely specify which semantic commitments must survive compression or how their preservation should be measured. We propose Context Codec, a commitment-level framework for compressing prompts and chat histories. Context Codec represents dialogue state as typed, source-grounded semantic atoms with canonical identity, equivalence, conflict, confidence, risk, and evidence spans. It separates five concerns - extraction, normalization, representation, rendering, and verification - and introduces metrics for Critical Atom Recall, Weighted Atom Recall, Commitment Density, and round-trip recoverability. It also defines a taxonomy of semantic compression errors, a concrete normalization procedure, conservative fallback rules for low-confidence and safety-critical atoms, and Context Compression Language (CCL), an ASCII-first compact rendering of canonical JSON atoms. In a small diagnostic study, CCL-Core occupies a useful middle ground between structured prose and JSON: more explicit and auditable than prose, usually more compact than JSON, and less risky than heavily minified notation. The result is not a claim that shorthand solves compression, but a framework for making context compression verifiable: compress the conversation, keep the commitments.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Prompt and context compression, one of 12 directions in Efficient LLM inference and serving. Builds on 15 of its 31 references that are in the map. First posted on arXiv 17 May 2026.
Builds on
15 of 31 references.
Closest ancestors · 3
Same direction, newest first
- 2024Prompt Compression for Large Language Models: A Survey16 Oct
- 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression19 Mar
- 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression10 Oct
Other references · 12
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
- 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 722
- 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 279
- 2023Compressing Context to Enhance Inference Efficiency of Large Language Modelscited by 185
- 2025A Survey of Context Engineering for Large Language Modelscited by 120
- 2025Uncertainty Quantification and Confidence Calibration in Large Language Models: A Surveycited by 93
- 2016Learning Executable Semantic Parsers for Natural Language Understandingcited by 21
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026SemanticZip: A Pilot Framework for Lossy Text Compression with LLMs as Semantic Decompressorscited by 0
- 2026What Does Context Compression Cost an Agent? Interaction Costs Unrevealed by Task-Completion Metricscited by 0
- 2026Towards Effective Structured Context Modeling for Conversational Recommender Systems via Dual-node Monte Carlo Tree Searchcited by 0
- 2026Towards Effective Structured Context Modeling for Conversational Recommender Systems via Dual-node Monte Carlo Tree Search1 Sep 2026
- 2026What Does Context Compression Cost an Agent? Interaction Costs Unrevealed by Task-Completion Metrics17 Aug 2026
- 2026SemanticZip: A Pilot Framework for Lossy Text Compression with LLMs as Semantic Decompressors23 May 2026
Similar, not cited
Close in wording, no citation either way
- 2025Fidelity Before Structure: Verbatim Chunks Beat Lossy Artifact Extraction in Long-Conversation LLM Memory
- 2026Developing Adaptive Context Compression Techniques for Large Language Models (LLMs) in Long-Running Interactions
- 2024Characterizing Prompt Compression Methods for Long Context Inference
- 2026MemRefine: LLM-Guided Compression for Long-Term Agent Memory
- 2026The Sleeping Agent: What Gist-Based Context Compression Loses and Why
- 2025From Context to EDUs: Faithful and Structured Context Compression via Elementary Discourse Unit Decomposition
- 2025LLM-Assisted Automated Deductive Coding of Dialogue Data: Leveraging Dialogue-Specific Characteristics to Enhance Contextual Understanding
- 2026LatentPress: Context Compression Beyond Text and Vision