mlsanityData to 7 Oct 2026

Discrete Prompt Compression with Reinforcement Learning

arXiv 2308.08758 ↗First posted 17 Aug 2023cs.CL cs.AIcited by 42

On the map: Efficient LLM inference and serving → Prompt and context compression

Abstract (arXiv)

Compressed prompts aid instruction-tuned language models (LMs) in overcoming context window limitations and reducing computational costs. Existing methods, which primarily based on training embeddings, face various challenges associated with interpretability, the fixed number of embedding tokens, reusability across different LMs, and inapplicability when interacting with black-box APIs. This study proposes prompt compression with reinforcement learning (PCRL), which is a discrete prompt compression method that addresses these issues. The proposed PCRL method utilizes a computationally efficient policy network that edits prompts directly. The training approach employed in the proposed PCRLs can be applied flexibly to various types of LMs, including both decoder-only and encoder-decoder architecture and it can be trained without gradient access to the LMs or labeled data. The proposed PCRL achieves an average reduction of 24.6% in terms of the token count across various instruction prompts while maintaining sufficient performance. In addition, we demonstrate that the learned policy can be transferred to larger LMs, and through a comprehensive analysis, we explore the token importance within the prompts. Our code is accessible at https://github.com/nenomigami/PromptCompressor.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Placed in Prompt and context compression, one of 12 directions in Efficient LLM inference and serving. Builds on 41 of its 55 references that are in the map. First posted on arXiv 17 Aug 2023.

Builds on

41 of 55 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Models6 Oct

Other references · 40

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  7. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  9. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  10. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  11. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  12. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
28 more
  1. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 849
  2. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  3. 2021GPT Understands, Toocited by 715
  4. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  5. 2016Self-critical Sequence Training for Image Captioningcited by 544
  6. 2019Reducing Transformer Depth on Demand with Structured Dropoutcited by 434
  7. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 420
  8. 2023Learning to Compress Prompts with Gist Tokenscited by 379
  9. 2017A Deep Reinforced Model for Abstractive Summarizationcited by 364
  10. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 361
  11. 2023Adapting Language Models to Compress Contextscited by 350
  12. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  13. 2020SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruningcited by 228
  14. 2023Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speechcited by 165
  15. 2022A Fast Post-Training Pruning Framework for Transformerscited by 161
  16. 2018Ranking Sentences for Extractive Summarization with Reinforcement Learningcited by 128
  17. 2022GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Modelscited by 127
  18. 2020On the Effect of Dropping Layers of Pre-trained Transformer Modelscited by 118
  19. 2023Unlocking Context Constraints of LLMs: Enhancing Context Efficiency of LLMs with Self-Information-Based Content Filteringcited by 62
  20. 2018Deep Communicating Agents for Abstractive Summarizationcited by 50
  21. 2018BanditSum: Extractive Summarization as a Contextual Banditcited by 46
  22. 2021Differentiable Subset Pruning of Transformer Headscited by 46
  23. 2021Keep it Simple: Unsupervised Simplification of Multi-Paragraph Textcited by 45
  24. 2020Stopwords in Technical Language Processingcited by 29
  25. 2021The Summary Loop: Learning to Write Abstractive Summaries Without Examplescited by 27
  26. 2020Discrete Optimization for Unsupervised Sentence Summarization with Word-Level Extractioncited by 23
  27. 2019Simple Unsupervised Summarization by Contextual Matchingcited by 18
  28. 2022Efficient Unsupervised Sentence Compression by Fine-tuning Transformers with Reinforcement Learningcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 15; 2025: 11; 2026: 14.

42 papers.

Sort:

By area:

  1. 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compressioncited by 343
  2. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  3. 2024A Survey on Efficient Inference for Large Language Modelscited by 197
  4. 2023Efficient Large Language Models: A Surveycited by 183
  5. 2024Prompt Compression for Large Language Models: A Surveycited by 65
  6. 2024Efficient Prompting Methods for Large Language Models: A Surveycited by 46
  7. 2024Symbolic Prompt Program Search: A Structure-Aware Approach to Efficient Compile-Time Prompt Optimizationcited by 29
  8. 2024TACO-RL: Task Aware Prompt Compression Optimization with Reinforcement Learningcited by 22
  9. 2024Prompt-SAW: Leveraging Relation-Aware Graphs for Textual Prompt Compressioncited by 18
  10. 2024Characterizing Prompt Compression Methods for Long Context Inferencecited by 18
  11. 2024MORL-Prompt: An Empirical Analysis of Multi-Objective Reinforcement Learning for Discrete Prompt Optimizationcited by 16
  12. 2025InfiniteICL: Breaking the Limit of Context Window Size via Long Short-term Memory Transformationcited by 16
30 more
  1. 2025Long Context In-Context Compression by Getting to the Gist of Gistingcited by 12
  2. 2024Semformer: Transformer Language Models with Semantic Planningcited by 9
  3. 2024Style-Compress: An LLM-Based Prompt Compression Framework Considering Task-Specific Stylescited by 9
  4. 2026Prompt Compression in the Wild: Measuring Latency, Rate Adherence, and Quality for Faster LLM Inferencecited by 8
  5. 2024From Reading to Compressing: Exploring the Multi-document Reader for Prompt Compressioncited by 7
  6. 2025DAC: A Dynamic Attention-aware Approach for Task-Agnostic Prompt Compressioncited by 7
  7. 2024Parse Trees Guided LLM Prompt Compressioncited by 6
  8. 2025Dynamic Compressing Prompts for Efficient Inference of Large Language Modelscited by 6
  9. 2025What Makes a Good Natural Language Prompt?cited by 6
  10. 2025EFPC: Towards Efficient and Flexible Prompt Compressioncited by 5
  11. 2025MOOSComp: Improving Lightweight Long-Context Compressor via Mitigating Over-Smoothing and Incorporating Outlier Scorescited by 4
  12. 2025Lossless Token Sequence Compression via Meta-Tokenscited by 4
  13. 2023MultiPrompter: Cooperative Prompt Optimization with Multi-Agent Reinforcement Learningcited by 3
  14. 2026ShortCoder: Knowledge-Augmented Syntax Optimization for Token-Efficient Code Generationcited by 3
  15. 2026Adapting Generalist Robot Policies with Semantic Reinforcement Learningcited by 3
  16. 2024TRIM: Token Reduction and Inference Modeling for Cost-Effective Language Generationcited by 2
  17. 2025CODEPROMPTZIP: Code-specific Prompt Compression for Retrieval-Augmented Generation in Coding Tasks with LMscited by 2
  18. 202698× Faster LLM Routing Without a Dedicated GPU: Flash Attention, Prompt Compression, and Near-Streaming for the vLLM Semantic Routercited by 2
  19. 2026BEAVER: A Training-Free Hierarchical Prompt Compression Method via Structure-Aware Page Selectioncited by 2
  20. 2026Compression Method Matters: Benchmark-Dependent Output Dynamics in LLM Prompt Compressioncited by 2
  21. 2026DiffuMask: Diffusion Language Model for Token-level Prompt Pruningcited by 1
  22. 2026IterCOMP: Reasoning-aware Adaptive Prompt Compression for Multi-hop Question Answeringcited by 1
  23. 2025Task-agnostic Prompt Compression with Context-aware Sentence Embedding and Reward-guided Task Descriptorcited by 0
  24. 2025When Compression Becomes an Attack Surface: Black-Box Attacks on Prompt-Compressed LLM Agentscited by 0
  25. 2026Developing Adaptive Context Compression Techniques for Large Language Models (LLMs) in Long-Running Interactionscited by 0
  26. 2026A Unified Model and Document Representation for On-Device Retrieval-Augmented Generationcited by 0
  27. 2026Compressing Sequences in the Latent Embedding Space: K-Token Merging for Large Language Modelscited by 0
  28. 2026Every Time I Hire a Linguist, Inference Costs Go Down: On Linguistic Rules as Effective Prompt Compressorscited by 0
  29. 2026Relevant but Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard Prompt Compressioncited by 0
  30. 2026Prompt Minimization: Reducing Input Redundancy Without Sacrificing Output Fidelitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026ZipRL: Adaptive Multi-Turn Context Compression with Hindsight Response Replay
  2. 2024LanguaShrink: Reducing Token Overhead with Psycholinguistics
  3. 2025PIS: Linking Importance Sampling and Attention Mechanisms for Efficient Prompt Compression
  4. 2026Prompt Compression via Activation Aggregation
  5. 2025Better Prompt Compression Without Multi-Layer Perceptrons
  6. 2025Understanding and Improving Information Preservation in Prompt Compression for LLMs
  7. 2025Learning a Single Token to Replace Long System Prompts in LLMs
  8. 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models