mlsanityData to 7 Oct 2026

Perception Compressor: A Training-Free Prompt Compression Framework in Long Context Scenarios

arXiv 2409.19272 ↗First posted 28 Sep 2024cs.CLcited by 22

On the map: Efficient LLM inference and serving → Prompt and context compression

Abstract (arXiv)

Large language models (LLMs) demonstrate exceptional capabilities in various scenarios. However, they suffer from much redundant information and are sensitive to the position of key information in long context scenarios. To address these challenges, we present Perception Compressor, a training-free prompt compression framework. It includes a perception retriever that leverages guiding questions and instruction to retrieve the most relevant demonstrations, a dual-slope ratio allocator to dynamically allocate compression ratios and open-book ratios, and a semi-guided iterative compression that retains key information at the token level while removing tokens that distract the LLM. We conduct extensive experiments on long context benchmarks, i.e., NaturalQuestions, LongBench, and MuSiQue. Experiment results show that Perception Compressor outperforms existing methods by a large margin, achieving state-of-the-art performance.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Placed in Prompt and context compression, one of 12 directions in Efficient LLM inference and serving. Builds on 19 of its 39 references that are in the map. First posted on arXiv 28 Sep 2024.

Builds on

19 of 39 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Context Embeddings for Efficient Answer Generation in RAG12 Jul
  2. 2024Retaining Key Information under High Compression Ratios: Query-Guided Compressor for LLMs4 Jun
  3. 2024xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token22 May
  4. 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression19 Mar
  5. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression10 Oct

Other references · 14

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  5. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  6. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  7. 2022A Survey on In-context Learningcited by 717
  8. 2023Adapting Language Models to Compress Contextscited by 350
  9. 2023RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentationcited by 253
  10. 2023Focused Transformer: Contrastive Training for Context Scalingcited by 133
  11. 2023Unlimiformer: Long-Range Transformers with Unlimited Length Inputcited by 127
  12. 2023Jina Embeddings 2: 8192-Token General-Purpose Text Embeddings for Long Documentscited by 103
2 more
  1. 2024Learning to Compress Prompt in Natural Language Formatscited by 54
  2. 2024Can Perplexity Reflect Large Language Model's Ability in Long Text Understanding?cited by 45

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 15.

22 papers.

Sort:

By area:

  1. 2026Natural-Language Agent Harnessescited by 54
  2. 2025DAST: Context-Aware Compression in LLMs via Dynamic Allocation of Soft Tokenscited by 21
  3. 2025GMSA: Enhancing Context Compression via Group Merging and Layer Semantic Alignmentcited by 20
  4. 2025Autoencoding-Free Context Compression for LLMs via Contextual Semantic Anchorscited by 14
  5. 2026COMI: Coarse-to-fine Context Compression via Marginal Information Gaincited by 13
  6. 2026Read As Human: Compressing Context via Parallelizable Close Reading and Skimmingcited by 10
  7. 2025CompLLM: Compression for Long Context Q&Acited by 5
  8. 2026CoMeT: Collaborative Memory Transformer for Efficient Long Context Modelingcited by 5
  9. 2026SemantiCache: Efficient KV Cache Compression via Semantic Chunking and Clustered Mergingcited by 5
  10. 2025From Token to Line: Enhancing Code Generation with a Long-Term Perspectivecited by 4
  11. 2026PoC: Performance-oriented Context Compression for Large Language Models via Performance Predictioncited by 3
  12. 2026Beyond Position Bias: Shifting Context Compression from Position-Driven to Semantic-Drivencited by 3
10 more
  1. 2025RAISE: Reinforced Adaptive Instruction Selection For Large Language Modelscited by 2
  2. 2026Length-Adaptive Interest Network for Balancing Long and Short Sequence Modeling in CTR Predictioncited by 2
  3. 2026Data Distribution Matters: A Data-Centric Perspective on Context Compression for Large Language Modelcited by 1
  4. 2026SAGE: A Quantitative Evaluation of Socialized Evolution in Agent Ecosystemscited by 1
  5. 2025PEMUTA: Pedagogically-Enriched Multi-Granular Undergraduate Thesis Assessmentcited by 0
  6. 2026When Hard Negatives Hurt: Bridging the Generative-Discriminative Gap in Hard Negative Synthesis for Retrievalcited by 0
  7. 2026HybridThinker: Efficient Chain-of-Thought Reasoning via Compressed Memory and Transient Thought Stepscited by 0
  8. 2026Mapping Text to Multiplex Graph: Prompt Compression as L'evy Walk-Guided Graph Pruningcited by 0
  9. 2026Relevant but Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard Prompt Compressioncited by 0
  10. 2026Compression Beyond the Uncompressed: A Two-Stage Training Recipe for Soft Context Compression in RAGcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Cmprsr: Abstractive Token-Level Question-Agnostic Prompt Compressor
  2. 2026Lost in Compression: A Controlled Cross-Lingual Audit of Extractive Prompt Compressors
  3. 2026One Token per Highly Selective Frame: Towards Extreme Compression for Long Video Understanding
  4. 2024Prompt Compression for Large Language Models: A Survey
  5. 2025Better Prompt Compression Without Multi-Layer Perceptrons
  6. 2024Enhancing and Accelerating Large Language Models via Instruction-Aware Contextual Compression
  7. 2026Cognitive Chunking for Soft Prompts: Accelerating Compressor Learning via Block-wise Causal Masking
  8. 2025Understanding and Improving Information Preservation in Prompt Compression for LLMs