mlsanityData to 7 Oct 2026

PCToolkit: A Unified Plug-and-Play Prompt Compression Toolkit of Large Language Models

arXiv 2403.17411 ↗First posted 26 Mar 2024cs.CLcited by 1

On the map: Efficient LLM inference and serving → Prompt and context compression

Abstract (arXiv)

Prompt compression is an innovative method for efficiently condensing input prompts while preserving essential information. To facilitate quick-start services, user-friendly interfaces, and compatibility with common datasets and metrics, we present the Prompt Compression Toolkit (PCToolkit). This toolkit is a unified plug-and-play solution for compressing prompts in Large Language Models (LLMs), featuring cutting-edge prompt compressors, diverse datasets, and metrics for comprehensive performance evaluation. PCToolkit boasts a modular design, allowing for easy integration of new datasets and metrics through portable and user-friendly interfaces. In this paper, we outline the key components and functionalities of PCToolkit. We conducted evaluations of the compressors within PCToolkit across various natural language tasks, including reconstruction, summarization, mathematical problem-solving, question answering, few-shot learning, synthetic tasks, code completion, boolean expressions, multiple choice questions, and lies recognition.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Prompt and context compression, one of 12 directions in Efficient LLM inference and serving. Builds on 23 of its 35 references that are in the map. First posted on arXiv 26 Mar 2024.

Builds on

23 of 35 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression10 Oct

Other references · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  4. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  6. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  7. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  8. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  9. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  10. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  11. 2018Self-Attention with Relative Position Representationscited by 1,112
  12. 2023Resurrecting Recurrent Neural Networks for Long Sequencescited by 397
10 more
  1. 2015A Neural Attention Model for Abstractive Sentence Summarizationcited by 394
  2. 2020IntelliCode Compose: Code Generation Using Transformercited by 225
  3. 2023Efficient Large Language Models: A Surveycited by 183
  4. 2021OpenPrompt: An Open-source Framework for Prompt-learningcited by 148
  5. 2023ChainForge: A Visual Toolkit for Prompt Engineering and LLM Hypothesis Testingcited by 88
  6. 2019Lightweight and Efficient End-to-End Speech Recognition Using Low-Rank Transformercited by 47
  7. 2021Keep it Simple: Unsupervised Simplification of Multi-Paragraph Textcited by 45
  8. 2024Prompt Design and Engineering: Introduction and Advanced Methodscited by 45
  9. 2021A Simple and Effective Positional Encoding for Transformerscited by 37
  10. 2022Efficient Unsupervised Sentence Compression by Fine-tuning Transformers with Reinforcement Learningcited by 18

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025ILRe: Intermediate Layer Retrieval for Context Compression in Causal Language Modelscited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2024Prompt Compression for Large Language Models: A Survey
  2. 2025Understanding and Improving Information Preservation in Prompt Compression for LLMs
  3. 2024Fundamental Limits of Prompt Compression: A Rate-Distortion Framework for Black-Box Language Models
  4. 2025Cmprsr: Abstractive Token-Level Question-Agnostic Prompt Compressor
  5. 2024Efficient Prompting Methods for Large Language Models: A Survey
  6. 2025An Empirical Study on Prompt Compression for Large Language Models
  7. 2026Lexical Prompt Compression for Large Language Models: A Training-Free, Deterministic Pipeline with Empirical Pareto Analysis Across Eleven Task Categories
  8. 2024Style-Compress: An LLM-Based Prompt Compression Framework Considering Task-Specific Styles