mlsanityData to 7 Oct 2026

Entropy Gate: Entropy Quenching for Near-Lossless Token Compression in LLM Pipelines

arXiv 2606.03739 ↗First posted 2 Jun 2026cs.CL cs.IT math.ITcited by 0

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

LLM pipelines waste substantial token budgets on low-information content: repeated context, verbose responses, and redundant boilerplate. We introduce Entropy Gate, a token compression framework applying entropy quenching - a thermodynamic process that progressively freezes out low-energy tokens while preserving semantic fidelity. Each token receives a multi-factor information energy E(t) combining statistical, structural, and positional components. An adaptive quenching schedule T(τ) = T0 / (1 + α τ) removes tokens whose Boltzmann survival probability pi = exp(-Ei / kT) falls below threshold, with a fidelity gate halting compression when energy-weighted similarity drops below θ. We prove token selection by descending E(t) maximizes expected semantic preservation, that quenching produces nested survival sets, and that achievable compression approaches the information-theoretic limit CR → 1 - I(P; T)/H(P). A Phase 1 heuristic achieves 40-60% compression across five prompt categories while maintaining SE > 0.80, with energy-squared amplification E → E2 adding 10-25 percentage points. Context deduplication adds 50-70% savings on repeated blocks. Output-side quenching, motivated by findings that brevity improves accuracy, further reduces response overhead. Combined with external memory, reduction composes multiplicatively to 88-96% for agentic workloads. The framework is stateless, model-agnostic, and deploys as an OpenAI-compatible HTTP proxy.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 21 of its 40 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

21 of 40 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs3 Oct
  2. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun

Other references · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2020Scaling Laws for Neural Language Modelscited by 6,411
  9. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  10. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  11. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  12. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
7 more
  1. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  2. 2020What is the State of Neural Network Pruning?cited by 578
  3. 2023Learning to Compress Prompts with Gist Tokenscited by 379
  4. 2023Adapting Language Models to Compress Contextscited by 350
  5. 2023LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Modelscited by 279
  6. 2023In-context Autoencoder for Context Compression in a Large Language Modelcited by 190
  7. 2026Brevity Constraints Reverse Performance Hierarchies in Language Modelscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Information-Aware KV Cache Compression for Long Reasoning
  2. 2025ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference
  3. 2024More Tokens, Lower Precision: Towards the Optimal Token-Precision Trade-off in KV Cache Compression
  4. 2025Cramming 1568 Tokens into a Single Vector and Back Again: Exploring the Limits of Embedding Space Capacity
  5. 2024Compressing KV Cache for Long-Context LLM Inference with Inter-Layer Attention Similarity
  6. 2026Fix the Structural Bottleneck: Context Compression via Explicit Information Transmission
  7. 2024Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference
  8. 2025Cache What Lasts: Token Retention for Memory-Bounded KV Cache in LLMs