mlsanityData to 7 Oct 2026

BEAVER: A Training-Free Hierarchical Prompt Compression Method via Structure-Aware Page Selection

arXiv 2603.19635 ↗First posted 20 Mar 2026cs.CLcited by 2

On the map: Efficient LLM inference and serving → Prompt and context compression

Abstract (arXiv)

The exponential expansion of context windows in LLMs has unlocked capabilities for long-document understanding but introduced severe bottlenecks in inference latency and information utilization. Existing compression methods often suffer from high training costs or semantic fragmentation due to aggressive token pruning. In this paper, we propose BEAVER, a novel training-free framework that shifts compression from linear token removal to structure-aware hierarchical selection. BEAVER maximizes hardware parallelism by mapping variable-length contexts into dense page-level tensors via dual-path pooling, and preserves discourse integrity through a hybrid planner combining semantic and lexical dual-branch selection with sentence smoothing. Extensive evaluations on four long-context benchmarks demonstrate that BEAVER achieves comparable performance to state-of-the-art (SOTA) methods like LongLLMLingua. Notably, on the RULER benchmark, BEAVER maintains high fidelity in multi-needle retrieval where baselines deteriorate. Regarding efficiency, BEAVER reduces latency by 26.4x on 128k contexts, offering a scalable solution for high-throughput applications. Our code is available at https://cslikai.cn/BEAVER/.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Prompt and context compression, one of 12 directions in Efficient LLM inference and serving. Builds on 24 of its 31 references that are in the map. First posted on arXiv 20 Mar 2026.

Builds on

24 of 31 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024TACO-RL: Task Aware Prompt Compression Optimization with Reinforcement Learning19 Sep
  2. 2024Prompt Compression with Context-Aware Sentence Encoding for Fast and Improved LLM Inference2 Sep
  3. 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression19 Mar
  4. 2023TCRA-LLM: Token Compression Retrieval Augmented Large Language Model for Inference Cost Reduction24 Oct
  5. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression10 Oct
  6. 2023Discrete Prompt Compression with Reinforcement Learning17 Aug

Other references · 18

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  4. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  6. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  7. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  8. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  9. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  10. 2021Efficient Attentions for Long Document Summarizationcited by 342
  11. 2023L-Eval: Instituting Standardized Evaluation for Long Context Language Modelscited by 232
  12. 2023LooGLE: Can Long-Context Language Models Understand Long Contexts?cited by 200
6 more
  1. 2023ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understandingcited by 175
  2. 2024MMLongBench-Doc: Benchmarking Long-context Document Understanding with Visualizationscited by 174
  3. 2025Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketchingcited by 116
  4. 2024LongDocURL: a Comprehensive Multimodal Long Document Benchmark Integrating Understanding, Reasoning, and Locatingcited by 70
  5. 2025The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMscited by 62
  6. 2025LONGCODEU: Benchmarking Long-Context Language Models on Long Code Understandingcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Relevant but Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard Prompt Compressioncited by 0
  2. 2026Multi-Agent Forensic Reasoning for Generalizable Deepfake Video Detectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026From Similarity to Structure: Training-free LLM Context Compression with Hybrid Graph Priors
  2. 2024Compressing KV Cache for Long-Context LLM Inference with Inter-Layer Attention Similarity
  3. 2025From Context to EDUs: Faithful and Structured Context Compression via Elementary Discourse Unit Decomposition
  4. 2026PuzzleKV: Page-Wise Low-Rank Decomposition for KV Cache Compression
  5. 2025CCF: A Context Compression Framework for Efficient Long-Sequence Language Modeling
  6. 2026StructKV: Preserving the Structural Skeleton for Scalable Long-Context Inference
  7. 2024Enhancing and Accelerating Large Language Models via Instruction-Aware Contextual Compression
  8. 2024Characterizing Prompt Compression Methods for Long Context Inference