mlsanityData to 7 Oct 2026

Understanding and Improving Information Preservation in Prompt Compression for LLMs

arXiv 2503.19114 ↗First posted 24 Mar 2025cs.CL cs.IR cs.LGcited by 13

On the map: Efficient LLM inference and serving → Prompt and context compression

Abstract (arXiv)

Recent advancements in large language models (LLMs) have enabled their successful application to a broad range of tasks. However, in information-intensive tasks, the prompt length can grow fast, leading to increased computational requirements, performance degradation, and induced biases from irrelevant or redundant information. Recently, various prompt compression techniques have been introduced to optimize the trade-off between reducing input length and retaining performance. We propose a holistic evaluation framework that allows for in-depth analysis of prompt compression methods. We focus on three key aspects, besides compression ratio: (i) downstream task performance, (ii) grounding in the input context, and (iii) information preservation. Using our framework, we analyze state-of-the-art soft and hard compression methods and show that some fail to preserve key details from the original prompt, limiting performance on complex tasks. By identifying these limitations, we are able to improve one soft prompting method by controlling compression granularity, achieving up to +23% in downstream performance, +8 BERTScore points in grounding, and 2.7x more entities preserved in compression. Ultimately, we find that the best effectiveness/compression rate trade-off is achieved with soft prompting combined with sequence-level training.The code is available at https://github.com/amazon-science/information-preservation-in-prompt-compression.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Placed in Prompt and context compression, one of 12 directions in Efficient LLM inference and serving. Builds on 36 of its 52 references that are in the map. First posted on arXiv 24 Mar 2025.

Builds on

36 of 52 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024From Reading to Compressing: Exploring the Multi-document Reader for Prompt Compression5 Oct
  2. 2024Fundamental Limits of Prompt Compression: A Rate-Distortion Framework for Black-Box Language Models22 Jul
  3. 2024xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token22 May
  4. 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression19 Mar
  5. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression10 Oct
  6. 2022Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Models6 Oct

Other references · 30

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  6. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  8. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  9. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  10. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  11. 2019How Can We Know What Language Models Know?cited by 953
  12. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
18 more
  1. 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 477
  2. 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 396
  3. 2018QuAC : Question Answering in Contextcited by 379
  4. 2023Learning to Compress Prompts with Gist Tokenscited by 379
  5. 2023Adapting Language Models to Compress Contextscited by 350
  6. 2022One Embedder, Any Task: Instruction-Finetuned Text Embeddingscited by 329
  7. 2024Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attentioncited by 194
  8. 2019Do NLP Models Know Numbers? Probing Numeracy in Embeddingscited by 157
  9. 2024FABLES: Evaluating faithfulness and content selection in book-length summarizationcited by 65
  10. 2023Unlocking Context Constraints of LLMs: Enhancing Context Efficiency of LLMs with Self-Information-Based Content Filteringcited by 62
  11. 2023Coverage-based Example Selection for In-Context Learningcited by 58
  12. 2024Learning to Compress Prompt in Natural Language Formatscited by 54
  13. 2023How Many Demonstrations Do You Need for In-context Learning?cited by 52
  14. 2023Extending Context Window of Large Language Models via Semantic Compressioncited by 45
  15. 2025The Amazon Nova Family of Models: Technical Report and Model Cardcited by 41
  16. 2023Did You Read the Instructions? Rethinking the Effectiveness of Task Definitions in Instruction Learningcited by 40
  17. 2023More Samples or More Prompts? Exploring Effective In-Context Sampling for LLM Few-Shot Prompt Engineeringcited by 13
  18. 2021Probing Pre-trained Auto-regressive Language Models for Named Entity Typing and Recognitioncited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 11.

13 papers.

Sort:

By area:

  1. 2025CLaRa: Bridging Retrieval and Generation with Continuous Latent Reasoningcited by 9
  2. 2025Beyond Hard and Soft: Hybrid Context Compression for Balancing Local and Global Information Retentioncited by 7
  3. 2026Cartridges at Scale: Training Modular KV Caches over Large Document Collectionscited by 6
  4. 202698× Faster LLM Routing Without a Dedicated GPU: Flash Attention, Prompt Compression, and Near-Streaming for the vLLM Semantic Routercited by 2
  5. 2026When Summaries Distort Decisions: Information Fidelity in LLM-Compressed Financial Analysiscited by 2
  6. 2026Fixed RAG Compression Collapses Measured Reader Scalingcited by 1
  7. 2026LLMs Underperform Graph-Based Parsers on Supervised Relation Extraction for Complex Graphscited by 0
  8. 2026When Less is Enough: Efficient Inference via Collaborative Reasoningcited by 0
  9. 2026Every Time I Hire a Linguist, Inference Costs Go Down: On Linguistic Rules as Effective Prompt Compressorscited by 0
  10. 2026Relevant but Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard Prompt Compressioncited by 0
  11. 2026Compression Beyond the Uncompressed: A Two-Stage Training Recipe for Soft Context Compression in RAGcited by 0
  12. 2026The Attribution-Compression Frontier in Retrieval-Augmented Generationcited by 0
1 more
  1. 2026Where Should a Document Live: Context, Representations, or Parameters?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Prompt Compression for Large Language Models: A Survey
  2. 2025An Empirical Study on Prompt Compression for Large Language Models
  3. 2026Lexical Prompt Compression for Large Language Models: A Training-Free, Deterministic Pipeline with Empirical Pareto Analysis Across Eleven Task Categories
  4. 2026Lost in Compression: A Controlled Cross-Lingual Audit of Extractive Prompt Compressors
  5. 2024Prompt-SAW: Leveraging Relation-Aware Graphs for Textual Prompt Compression
  6. 2026Prompt Compression via Activation Aggregation
  7. 2026Prompt Minimization: Reducing Input Redundancy Without Sacrificing Output Fidelity
  8. 2026Prompt Compression in the Wild: Measuring Latency, Rate Adherence, and Quality for Faster LLM Inference