# Cmprsr: Abstractive Token-Level Question-Agnostic Prompt Compressor

arXiv: https://arxiv.org/abs/2511.12281 · HTML: https://mlsanity.com/paper/2511.12281/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Prompt and context compression, one of 12 directions in Efficient LLM inference and serving.
- Builds on 22 of its 43 references that are in the map.
- First posted on arXiv 15 Nov 2025.

## Placed in

- Efficient LLM inference and serving / [Prompt and context compression](https://mlsanity.com/inference/prompt-and-context-compression/)

## Builds on (most cited first, 22 of 22)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://mlsanity.com/paper/2312.10997/) (2023)
- [Learning to Compress Prompts with Gist Tokens](https://mlsanity.com/paper/2304.08467/) (2023)
- [Adapting Language Models to Compress Contexts](https://mlsanity.com/paper/2305.14788/) (2023)
- [LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression](https://mlsanity.com/paper/2403.12968/) (2024)
- [Learning Performance-Improving Code Edits](https://mlsanity.com/paper/2302.07867/) (2023)
- [MeetingBank: A Benchmark Dataset for Meeting Summarization](https://mlsanity.com/paper/2305.17529/) (2023)
- [Prompt Compression for Large Language Models: A Survey](https://mlsanity.com/paper/2410.12388/) (2024)
- [CompAct: Compressing Retrieved Documents Actively for Question Answering](https://mlsanity.com/paper/2407.09014/) (2024)
- [The Wisdom of Hindsight Makes Language Models Better Instruction Followers](https://mlsanity.com/paper/2302.05206/) (2023)
- [Learning to Compress Prompt in Natural Language Formats](https://mlsanity.com/paper/2402.18700/) (2024)
- [Prompt Compression with Context-Aware Sentence Encoding for Fast and Improved LLM Inference](https://mlsanity.com/paper/2409.01227/) (2024)
- [The Widespread Adoption of Large Language Model-Assisted Writing Across Society](https://mlsanity.com/paper/2502.09747/) (2025)
- [From Efficiency Gains to Rebound Effects: The Problem of Jevons' Paradox in AI's Polarized Environmental Debate](https://mlsanity.com/paper/2501.16548/) (2025)
- [TACO-RL: Task Aware Prompt Compression Optimization with Reinforcement Learning](https://mlsanity.com/paper/2409.13035/) (2024)
- [Style-Compress: An LLM-Based Prompt Compression Framework Considering Task-Specific Styles](https://mlsanity.com/paper/2410.14042/) (2024)
- [From Reading to Compressing: Exploring the Multi-document Reader for Prompt Compression](https://mlsanity.com/paper/2410.04139/) (2024)
- [DAC: A Dynamic Attention-aware Approach for Task-Agnostic Prompt Compression](https://mlsanity.com/paper/2507.11942/) (2025)
- [Dynamic Compressing Prompts for Efficient Inference of Large Language Models](https://mlsanity.com/paper/2504.11004/) (2025)
- [SCOPE: A Generative Approach for LLM Prompt Compression](https://mlsanity.com/paper/2508.15813/) (2025)
- [PromptOptMe: Error-Aware Prompt Compression for LLM-based MT Evaluation Metrics](https://mlsanity.com/paper/2412.16120/) (2024)
- [ACoRN: Noise-Robust Abstractive Compression in Retrieval-Augmented Language Models](https://mlsanity.com/paper/2504.12673/) (2025)

## Built on by (most cited first, 1 of 1)

- [Distilling Sequential Computation in Transformer Language Models](https://mlsanity.com/paper/2609.27233/) (2026)

## Abstract (arXiv)

Motivated by the high costs of using black-box Large Language Models (LLMs), we introduce a novel prompt compression paradigm, under which we use smaller LLMs to compress inputs for the larger ones. We present the first comprehensive LLM-as-a-compressor benchmark spanning 25 open- and closed-source models, which reveals significant disparity in models' compression ability in terms of (i) preserving semantically important information (ii) following the user-provided compression rate (CR). We further improve the performance of gpt-4.1-mini, the best overall vanilla compressor, with Textgrad-based compression meta-prompt optimization. We also identify the most promising open-source vanilla LLM - Qwen3-4B - and post-train it with a combination of supervised fine-tuning (SFT) and Group Relative Policy Optimization (GRPO), pursuing the dual objective of CR adherence and maximizing the downstream task performance. We call the resulting model Cmprsr and demonstrate its superiority over both extractive and vanilla abstractive compression across the entire range of compression rates on lengthy inputs from MeetingBank and LongBench as well as short prompts from GSM8k. The latter highlights Cmprsr's generalizability across varying input lengths and domains. Moreover, Cmprsr closely follows the requested compression rate, offering fine control over the cost-quality trade-off.