mlsanityData to 14 Oct 2026

APCE: Adaptive Progressive Context Expansion for Long Context Processing

arXiv 2510.12051 ↗First posted 14 Oct 2025cs.CL cs.AIcited by 0

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Deploying useful Long-Context Transformer Models (LCTMs) requires addressing two key challenges: (1) A growing memory footprint due to quadratic self-attention and linear KV-cache scaling in memory as sequence length increases; (2) the ContextRot phenomena where empirical evidence suggests that transformer architecture's performance degrades with increasing context length. Given the shared dependency on the input, a natural question arises: Can we surgically select the most important input chunks for processing to synergistically (a) reduce the memory footprint, and (b) mitigate the ContextRot effects? In this paper, we answer this question in the affirmative for long-context summarization tasks. We propose APCE as a context-aware solution to select the most important input chunks through low-dimensional semantic similarity matching with the current query. By directly operating on the input, APCE decouples from strict dependency on underlying hardware or CUDA environments, promising a compatible solution scalable to different deployment systems. Our empirical evaluations have demonstrated superior or on-par summarization performance for APCE compared to the full dense baseline using a fraction (50%-70%) of the input sequence resulting in KV-cache and self-attention memory efficiency improvements. We hope our findings inspire further research on context-aware efficiency solutions for LCTMs geared towards other relevant long-context tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 18 of its 22 references that are in the map. First posted on arXiv 14 Oct 2025.

Builds on

18 of 22 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025LLMs Know What to Drop: Self-Attention Guided KV Cache Eviction for Efficient Long-Context Inference11 Mar
  2. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference16 Jun
  3. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun

Other references · 15

  1. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  3. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  4. 2020Longformer: The Long-Document Transformercited by 3,413
  5. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  6. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  7. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  8. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  9. 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 383
  10. 2020SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruningcited by 234
  11. 2021BookSum: A Collection of Datasets for Long-form Narrative Summarizationcited by 184
  12. 2025XAttention: Block Sparse Attention with Antidiagonal Scoringcited by 137
3 more
  1. 2025FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inferencecited by 123
  2. 2024Transformers are Multi-State RNNscited by 120
  3. 2024Training-Free Long-Context Scaling of Large Language Modelscited by 62

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Compress, Gather, and Recompute: REFORMing Long-Context Processing in Transformers
  2. 2025A Survey on Transformer Context Extension: Approaches and Evaluation
  3. 2026Latent-Condensed Transformer for Efficient Long Context Modeling
  4. 2026Training Long-Context Vision-Language Models Effectively with Generalization Beyond 128K Context
  5. 2024Enhancing and Accelerating Large Language Models via Instruction-Aware Contextual Compression
  6. 2024Hierarchical Context Merging: Better Long Context Understanding for Pre-trained LLMs
  7. 2025Context Discipline and Performance Correlation: Analyzing LLM Performance and Quality Degradation Under Varying Context Lengths
  8. 2024Recurrent Context Compression: Efficiently Expanding the Context Window of LLM