# When Less is More: The LLM Scaling Paradox in Context Compression

arXiv: https://arxiv.org/abs/2602.09789 · HTML: https://mlsanity.com/paper/2602.09789/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Prompt and context compression, one of 12 directions in Efficient LLM inference and serving.
- First posted on arXiv 10 Feb 2026.

## Placed in

- Efficient LLM inference and serving / [Prompt and context compression](https://mlsanity.com/inference/prompt-and-context-compression/)

## Built on by (most cited first, 1 of 1)

- [ReSum: Synergizing LLM Reasoning and Summarization with Reinforcement Learning](https://mlsanity.com/paper/2606.13316/) (2026)

## Abstract (arXiv)

Scaling up model parameters has long been a prevalent training paradigm driven by the assumption that larger models yield superior generation capabilities. However, under lossy context compression in a compressor--decoder setup, we find a textbftextitSize-Fidelity Paradox: increasing compressor size can lessen the faithfulness of reconstructed contexts though reconstruction error decreases. Across 27 compressor setups spanning model families, scales, and compression rates, we coin this paradox arising from two dominant factors: 1) textitknowledge overwriting: larger models increasingly replace source facts with their own prior beliefs, textite.g., ``the white strawberry`` → ``the red strawberry``; and 2) textitsemantic drift: larger models tend to paraphrase or restructure content instead of reproducing it verbatim, textite.g., ``Alice hit Bob`` → ``Bob hit Alice``. Interestingly, this paradox persists across varied settings, with mid-sized compressors often outperforming larger ones in faithful recovery. By analyzing the compressed memory via embedding geometry and reconstruction determinacy, we further reveal that compressors tend to organize memory across broader semantic subspaces, yielding more ambiguous representations prone to overwriting, drift, and weakened recovery. These findings complement existing evaluations of context compression and expose a breakdown of scaling laws when the objective shifts from plausible generation to faithful preservation.