mlsanityData to 7 Oct 2026

The Compression Paradox in LLM Inference: Provider-Dependent Energy Effects of Prompt Compression

arXiv 2603.23528 ↗First posted 6 Mar 2026cs.CLcited by 0

On the map: Efficient LLM inference and serving → Prompt and context compression

Abstract (arXiv)

The rapid proliferation of Large Language Models has created an environmental paradox: the very technology that could help solve climate challenges is itself becoming a significant contributor to global carbon emissions. We test whether prompt compression improves inference energy efficiency in 28,421 successful API trials (28,428 planned) across three providers (OpenAI GPT-4o-mini, Anthropic Claude-3.5-Sonnet, and DeepSeek-Chat), five benchmarks (HumanEval, MBPP, GSM8K, MATH, MMLU), and four compression ratios (r in {1.0, 0.7, 0.5, 0.3}). Energy is estimated with a token-based proxy calibrated against local direct measurements, and quality is tracked with benchmark pass rates. Compression produced substantial quality loss (overall pass rate 26.0% at baseline vs. 1.5% at r=0.7) and strongly provider-dependent energy behavior. DeepSeek exhibited output expansion under compression (21 to 798 tokens at r=0.3), corresponding to energy increases up to +2,140%, while GPT-4o-mini showed mixed effects including a reduction at r=0.5. These results indicate that input-token reduction alone is not a reliable energy optimization strategy in production inference. For the evaluated settings, model selection and output-length control provided more consistent energy-quality tradeoffs than prompt compression.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Prompt and context compression, one of 12 directions in Efficient LLM inference and serving. Builds on 23 of its 35 references that are in the map. First posted on arXiv 6 Mar 2026.

Builds on

23 of 35 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression19 Mar
  2. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression10 Oct

Other references · 21

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  5. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  6. 2021Program Synthesis with Large Language Modelscited by 3,509
  7. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  8. 2019Green AIcited by 620
  9. 2021Carbon Emissions and Large Neural Network Trainingcited by 561
  10. 2019Quantifying the Carbon Emissions of Machine Learningcited by 516
  11. 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 279
  12. 2022Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Modelcited by 264
9 more
  1. 2020Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learningcited by 246
  2. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  3. 2023Power Hungry Processing: Watts Driving the Cost of AI Deployment?cited by 186
  4. 2022The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrinkcited by 183
  5. 2023LLMCarbon: Modeling the end-to-end Carbon Footprint of Large Language Modelscited by 71
  6. 2019Energy Usage Reports: Environmental awareness as part of algorithmic accountabilitycited by 43
  7. 2021Carbon-Aware Computing for Datacenterscited by 40
  8. 2025TokenPowerBench: Benchmarking the Power Consumption of LLM Inferencecited by 21
  9. 2026The Perplexity Paradox: Why Code Compresses Better Than Math in LLM Promptscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Prompt Compression in the Wild: Measuring Latency, Rate Adherence, and Quality for Faster LLM Inference
  2. 2026The Energy Footprint of LLM-Based Environmental Analysis: LLMs and Domain Products
  3. 2026When Less is More: The LLM Scaling Paradox in Context Compression
  4. 2025Information Capacity: Evaluating the Efficiency of Large Language Models via Text Compression
  5. 2024Faster and Lighter LLMs: A Survey on Current Challenges and Way Forward
  6. 2026Compression Method Matters: Benchmark-Dependent Output Dynamics in LLM Prompt Compression
  7. 2025Optimizing LLMs for Resource-Constrained Environments: A Survey of Model Compression Techniques
  8. 2026A Picture is Worth a Thousand Tokens: How Vision Language Models Cut AI Energy Costs While Improving Accuracy