Lost in Compression: A Controlled Cross-Lingual Audit of Extractive Prompt Compressors
On the map: Efficient LLM inference and serving → Prompt and context compression
Abstract (arXiv)
Extractive prompt compression promises to cut LLM inference costs by removing low-information tokens, and learned compressors such as LLMLingua-2 report strong results on English benchmarks. Most other languages already pay a token premium: the same content costs 1.3-1.8x more tokens than in English. We ask whether compression closes or widens this gap. Using fully parallel data in ten languages spanning five scripts, with controls budget-matched in the target model's tokenizer, we audit four learned compressors against four deterministic baselines, on eleven target models from ten vendors (over 250,000 evaluation calls). Three of the compressors are trained with English supervision (LLMLingua-2 XLM-R/mBERT; Kompress-v2 from the production Headroom stack); the fourth, XProvence, is trained multilingually. First, the transfer gap is real, replicates across target models and compressor backbones, and is strongly rate-dependent: at a 0.33 keep-rate English retains 57-62% of normalized context utilization while Lithuanian retains 10-24% and Chinese essentially none, despite Chinese having the smallest token premium. Second, the gap tracks compression supervision data, not architecture. All three English-trained compressors show it, deterministic methods show no comparable gap, and the multilingually trained XProvence v1 shows none. Its v2 release, retrained on translated data, empties 92% of Chinese contexts at its aggressive threshold without any warning. Third, in a harder long-context setting, aggressive learned compression drives compressed contexts to or below no-context utility in three of five non-English languages. A translate-then-compress pipeline matches or beats native compression at roughly half the token cost in three of five tested languages. We release all code, compressions, and model outputs. Safe compression budgets are much smaller outside English.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Prompt and context compression, one of 12 directions in Efficient LLM inference and serving. Builds on 22 of its 23 references that are in the map. First posted on arXiv 27 Jul 2026.
Builds on
22 of 23 references.
Closest ancestors · 9
Same direction, newest first
- 2026Prompt Compression in Diffusion Large Language Models: Evaluating LLMLingua-2 on LLaDA18 May
- 2026Prompt Compression in the Wild: Measuring Latency, Rate Adherence, and Quality for Faster LLM Inference3 Apr
- 2025Sentinel: Decoding Context Utilization via Attention Probing for Efficient LLM Context Compression29 May
- 2024EXIT: Context-Aware Extractive Compression for Enhancing Retrieval-Augmented Generation17 Dec
- 2024Prompt Compression for Large Language Models: A Survey16 Oct
- 2024Prompt Compression with Context-Aware Sentence Encoding for Fast and Improved LLM Inference2 Sep
- 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression19 Mar
- 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression10 Oct
- 2023In-context Autoencoder for Context Compression in a Large Language Model13 Jul
Other references · 13
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
- 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
- 2023Learning to Compress Prompts with Gist Tokenscited by 379
- 2023The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variantscited by 300
- 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 279
- 2023RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentationcited by 253
- 2023Language Model Tokenizers Introduce Unfairness Between Languagescited by 207
- 2023Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Modelscited by 186
- 2023Compressing Context to Enhance Inference Efficiency of Large Language Modelscited by 185
- 2021MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfercited by 89
- 2025Provence: efficient and robust context pruning for retrieval-augmented generationcited by 25
- 2026XProvence: Zero-Cost Multilingual Context Pruning for Retrieval-Augmented Generationcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Understanding and Improving Information Preservation in Prompt Compression for LLMs
- 2026Every Time I Hire a Linguist, Inference Costs Go Down: On Linguistic Rules as Effective Prompt Compressors
- 2026Lexical Prompt Compression for Large Language Models: A Training-Free, Deterministic Pipeline with Empirical Pareto Analysis Across Eleven Task Categories
- 2026CAVEWOMAN: How Large Language Models Behave Under Linguistic Input and Output Compression
- 2026SemanticFold: Latent Sequence Compression SeparatesLanguage Modeling, Decodability, and Reasoning
- 2024LanguaShrink: Reducing Token Overhead with Psycholinguistics
- 2025Can Compressed LLMs Truly Act? An Empirical Evaluation of Agentic Capabilities in LLM Compression
- 2025Cmprsr: Abstractive Token-Level Question-Agnostic Prompt Compressor