# PISCO: Pretty Simple Compression for Retrieval-Augmented Generation

arXiv: https://arxiv.org/abs/2501.16075 · HTML: https://mlsanity.com/paper/2501.16075/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation.
- Builds on 30 of its 44 references that are in the map.
- First posted on arXiv 27 Jan 2025.

## Placed in

- Retrieval-augmented generation / [Generation and context integration](https://mlsanity.com/rag/generation-and-context-integration/)

## Builds on (most cited first, 30 of 30)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing](https://mlsanity.com/paper/2111.09543/) (2021)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators](https://mlsanity.com/paper/2404.04475/) (2024)
- [Sequence-Level Knowledge Distillation](https://mlsanity.com/paper/1606.07947/) (2016)
- [KILT: a Benchmark for Knowledge Intensive Language Tasks](https://mlsanity.com/paper/2009.02252/) (2020)
- [Adapting Language Models to Compress Contexts](https://mlsanity.com/paper/2305.14788/) (2023)
- [LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression](https://mlsanity.com/paper/2403.12968/) (2024)
- [Large Language Models Are Reasoning Teachers](https://mlsanity.com/paper/2212.10071/) (2022)
- [RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation](https://mlsanity.com/paper/2310.04408/) (2023)
- [ASQA: Factoid Questions Meet Long-Form Answers](https://mlsanity.com/paper/2204.06092/) (2022)
- [In-context Autoencoder for Context Compression in a Large Language Model](https://mlsanity.com/paper/2307.06945/) (2023)
- [Compressing Context to Enhance Inference Efficiency of Large Language Models](https://mlsanity.com/paper/2310.06201/) (2023)
- [xRAG: Extreme Context Compression for Retrieval-augmented Generation with One Token](https://mlsanity.com/paper/2405.13792/) (2024)
- [Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG](https://mlsanity.com/paper/2410.05983/) (2024)
- [Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Models](https://mlsanity.com/paper/2210.03162/) (2022)
- [Long-Context Language Modeling with Parallel Context Encoding](https://mlsanity.com/paper/2402.16617/) (2024)
- [Learning to Filter Context for Retrieval-Augmented Generation](https://mlsanity.com/paper/2311.08377/) (2023)
- [SPLADE-v3: New baselines for SPLADE](https://mlsanity.com/paper/2403.06789/) (2024)
- [DuoRC: Towards Complex Language Understanding with Paraphrased Reading Comprehension](https://mlsanity.com/paper/1804.07927/) (2018)
- [Inference Scaling for Long-Context Retrieval Augmented Generation](https://mlsanity.com/paper/2410.04343/) (2024)
- [500xCompressor: Generalized Prompt Compression for Large Language Models](https://mlsanity.com/paper/2408.03094/) (2024)
- [BERGEN: A Benchmarking Library for Retrieval-Augmented Generation](https://mlsanity.com/paper/2407.01102/) (2024)
- [Context Embeddings for Efficient Answer Generation in RAG](https://mlsanity.com/paper/2407.09252/) (2024)
- [Stronger Models are NOT Stronger Teachers for Instruction Tuning](https://mlsanity.com/paper/2411.07133/) (2024)
- [Zero-shot cross-lingual transfer in instruction tuning of large language models](https://mlsanity.com/paper/2402.14778/) (2024)

## Built on by (most cited first, 20 of 20)

- [Reducing Cost of LLM Agents with Trajectory Reduction](https://mlsanity.com/paper/2509.23586/) (2025)
- [CLaRa: Bridging Retrieval and Generation with Continuous Latent Reasoning](https://mlsanity.com/paper/2511.18659/) (2025)
- [SARA: Selective and Adaptive Retrieval-augmented Generation with Context Compression](https://mlsanity.com/paper/2507.05633/) (2025)
- [Multi-Vector Index Compression in Any Modality](https://mlsanity.com/paper/2602.21202/) (2026)
- [No Mean Feat: Simple, Strong Baselines for Context Compression](https://mlsanity.com/paper/2510.20797/) (2025)
- [OSCAR: Online Soft Compression And Reranking](https://mlsanity.com/paper/2504.07109/) (2025)
- [Adapting Large Language Models for Multi-Domain Retrieval-Augmented-Generation](https://mlsanity.com/paper/2504.02411/) (2025)
- [Privacy-Preserving Reasoning with Knowledge-Distilled Parametric Retrieval Augmented Generation](https://mlsanity.com/paper/2509.01088/) (2025)
- [ARC-Encoder: learning compressed text representations for large language models](https://mlsanity.com/paper/2510.20535/) (2025)
- [Less Is More: Elevating RAG via Performance-Driven Context Compression](https://mlsanity.com/paper/2508.19282/) (2025)
- [When More Retrieval Hurts: Retrieval-Augmented Code Review Generation](https://mlsanity.com/paper/2511.05302/) (2025)
- [XProvence: Zero-Cost Multilingual Context Pruning for Retrieval-Augmented Generation](https://mlsanity.com/paper/2601.18886/) (2026)
- [Cognitive Chunking for Soft Prompts: Accelerating Compressor Learning via Block-wise Causal Masking](https://mlsanity.com/paper/2602.13980/) (2026)
- [Grounded Cache Routing for Retrieval-Augmented Generation: When Is It Safe to Reuse an Answer?](https://mlsanity.com/paper/2605.27494/) (2026)
- [Fixed RAG Compression Collapses Measured Reader Scaling](https://mlsanity.com/paper/2606.21807/) (2026)
- [A Unified Model and Document Representation for On-Device Retrieval-Augmented Generation](https://mlsanity.com/paper/2604.14403/) (2026)
- [RAGOCR: Optical Compression of Retrieval-Augmented Text via Visual Representation](https://mlsanity.com/paper/2608.00765/) (2026)
- [Compression Beyond the Uncompressed: A Two-Stage Training Recipe for Soft Context Compression in RAG](https://mlsanity.com/paper/2609.05152/) (2026)
- [REVA: Reusable Evidence View Aggregation for Context-Efficient RAG Serving](https://mlsanity.com/paper/2609.11209/) (2026)
- [BoundaryMORPH: Budgeted Reranking via Active Set Selection for Diffuse Retrieval](https://mlsanity.com/paper/2609.27213/) (2026)

## Abstract (arXiv)

Retrieval-Augmented Generation (RAG) pipelines enhance Large Language Models (LLMs) by retrieving relevant documents, but they face scalability issues due to high inference costs and limited context size. Document compression is a practical solution, but current soft compression methods suffer from accuracy losses and require extensive pretraining. In this paper, we introduce PISCO, a novel method that achieves a 16x compression rate with minimal accuracy loss (0-3%) across diverse RAG-based question-answering (QA) tasks. Unlike existing approaches, PISCO requires no pretraining or annotated data, relying solely on sequence-level knowledge distillation from document-based questions. With the ability to fine-tune a 7-10B LLM in 48 hours on a single A100 GPU, PISCO offers a highly efficient and scalable solution. We present comprehensive experiments showing that PISCO outperforms existing compression models by 8% in accuracy.