# Selection-p: Self-Supervised Task-Agnostic Prompt Compression for Faithfulness and Transferability

arXiv: https://arxiv.org/abs/2410.11786 · HTML: https://mlsanity.com/paper/2410.11786/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Prompt and context compression, one of 12 directions in Efficient LLM inference and serving.
- Builds on 16 of its 26 references that are in the map.
- First posted on arXiv 15 Oct 2024.

## Placed in

- Efficient LLM inference and serving / [Prompt and context compression](https://mlsanity.com/inference/prompt-and-context-compression/)

## Builds on (most cited first, 16 of 16)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [Token Merging: Your ViT But Faster](https://mlsanity.com/paper/2210.09461/) (2022)
- [DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification](https://mlsanity.com/paper/2106.02034/) (2021)
- [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020)
- [Learning to Compress Prompts with Gist Tokens](https://mlsanity.com/paper/2304.08467/) (2023)
- [Adapting Language Models to Compress Contexts](https://mlsanity.com/paper/2305.14788/) (2023)
- [LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression](https://mlsanity.com/paper/2403.12968/) (2024)
- [Long-context LLMs Struggle with Long In-context Learning](https://mlsanity.com/paper/2404.02060/) (2024)
- [Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning](https://mlsanity.com/paper/2305.14160/) (2023)
- [Noisy Channel Language Model Prompting for Few-Shot Text Classification](https://mlsanity.com/paper/2108.04106/) (2021)
- [Focused Transformer: Contrastive Training for Context Scaling](https://mlsanity.com/paper/2307.03170/) (2023)
- [Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Models](https://mlsanity.com/paper/2210.03162/) (2022)
- [Length-Adaptive Transformer: Train Once with Length Drop, Use Anytime with Search](https://mlsanity.com/paper/2010.07003/) (2020)
- [AdapLeR: Speeding up Inference by Adaptive Length Reduction](https://mlsanity.com/paper/2203.08991/) (2022)

## Built on by (most cited first, 9 of 9)

- [Beyond Hard and Soft: Hybrid Context Compression for Balancing Local and Global Information Retention](https://mlsanity.com/paper/2505.15774/) (2025)
- [ProCut: LLM Prompt Compression via Attribution Estimation](https://mlsanity.com/paper/2508.02053/) (2025)
- [CompLLM: Compression for Long Context Q&A](https://mlsanity.com/paper/2509.19228/) (2025)
- [Learning What Not to Forget: Long-Horizon Agent Memory from a Few Kilobytes of Learning](https://mlsanity.com/paper/2606.20954/) (2026)
- [Many-Shot CoT-ICL: Making In-Context Learning Truly Learn](https://mlsanity.com/paper/2605.13511/) (2026)
- [A New Role for Relevance: Guiding Corpus Interaction in Agentic Search](https://mlsanity.com/paper/2607.24223/) (2026)
- [Every Time I Hire a Linguist, Inference Costs Go Down: On Linguistic Rules as Effective Prompt Compressors](https://mlsanity.com/paper/2607.25335/) (2026)
- [InsightEmb: Learning Action-Intent Embeddings for Agentic Insight Retrieval](https://mlsanity.com/paper/2608.04761/) (2026)
- [Compression Beyond the Uncompressed: A Two-Stage Training Recipe for Soft Context Compression in RAG](https://mlsanity.com/paper/2609.05152/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated impressive capabilities in a wide range of natural language processing tasks when leveraging in-context learning. To mitigate the additional computational and financial costs associated with in-context learning, several prompt compression methods have been proposed to compress the in-context learning prompts. Despite their success, these methods face challenges with transferability due to model-specific compression, or rely on external training data, such as GPT-4. In this paper, we investigate the ability of LLMs to develop a unified compression method that discretizes uninformative tokens, utilizing a self-supervised pre-training technique. By introducing a small number of parameters during the continual pre-training, the proposed Selection-p produces a probability for each input token, indicating whether to preserve or discard it. Experiments show Selection-p achieves state-of-the-art performance across numerous classification tasks, achieving compression rates of up to 10 times while experiencing only a marginal 0.8% decrease in performance. Moreover, it exhibits superior transferability to different models compared to prior work. Additionally, we further analyze how Selection-p helps maintain performance on in-context learning with long contexts.