mlsanityData to 14 Oct 2026

Developing Adaptive Context Compression Techniques for Large Language Models (LLMs) in Long-Running Interactions

arXiv 2603.29193 ↗First posted 31 Mar 2026cs.CV cs.AIcited by 0

On the map: Efficient LLM inference and serving → Prompt and context compression

Abstract (arXiv)

Large Language Models (LLMs) often experience performance degradation during long-running interactions due to increasing context length, memory saturation, and computational overhead. This paper presents an adaptive context compression framework that integrates importance-aware memory selection, coherence-sensitive filtering, and dynamic budget allocation to retain essential conversational information while controlling context growth. The approach is evaluated on LOCOMO, LOCCO, and LongBench benchmarks to assess answer quality, retrieval accuracy, coherence preservation, and efficiency. Experimental results demonstrate that the proposed method achieves consistent improvements in conversational stability and retrieval performance while reducing token usage and inference latency compared with existing memory and compression-based approaches. These findings indicate that adaptive context compression provides an effective balance between long-term memory preservation and computational efficiency in persistent LLM interactions

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Prompt and context compression, one of 12 directions in Efficient LLM inference and serving. Builds on 11 of its 23 references that are in the map. First posted on arXiv 31 Mar 2026.

Builds on

11 of 23 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Discrete Prompt Compression with Reinforcement Learning17 Aug

Other references · 10

  1. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  2. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  3. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
  4. 2024InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memorycited by 197
  5. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 135
  6. 2025Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generationcited by 36
  7. 2025Hierarchical Document Refinement for Long-context Retrieval-augmented Generationcited by 16
  8. 2025LAVa: Layer-wise KV Cache Eviction with Dynamic Budget Allocationcited by 16
  9. 2024In-Context Former: Lightning-fast Compressing Context for Large Language Modelcited by 12
  10. 2026ATACompressor: Adaptive Task-Aware Compression for Efficient Long-Context Processing in LLMscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Enhancing and Accelerating Large Language Models via Instruction-Aware Contextual Compression
  2. 2026MemRefine: LLM-Guided Compression for Long-Term Agent Memory
  3. 2025CCF: A Context Compression Framework for Efficient Long-Sequence Language Modeling
  4. 2024Recurrent Context Compression: Efficiently Expanding the Context Window of LLM
  5. 2026Compressing Long Context into Answer-Aligned Memory Embeddings for LLM Inference
  6. 2025AttnComp: Attention-Guided Adaptive Context Compression for Retrieval-Augmented Generation
  7. 2026PoC: Performance-oriented Context Compression for Large Language Models via Performance Prediction
  8. 2026Context-Driven Incremental Compression for Multi-Turn Dialogue Generation