mlsanityData to 7 Oct 2026

EntroLLM: Entropy Encoded Weight Compression for Efficient Large Language Model Inference on Edge Devices

arXiv 2505.02380 ↗First posted 5 May 2025cs.LGcited by 1

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Large Language Models (LLMs) achieve strong performance across tasks, but face storage and compute challenges on edge devices. We propose EntroLLM, a compression framework combining mixed quantization and entropy coding to reduce storage while preserving accuracy. We use a combination of unsigned and asymmetric quantization. Tensor-level quantization produces an entropy-reducing effect, increasing weight compressibility, and improving downstream Huffman encoding by 7× (8-bit) and 11.3× (4-bit) over state-of-the-art methods. Huffman coding further reduces memory bandwidth demands, while a parallel decoding strategy enables efficient weight retrieval with minimal latency. Experiments on edge-scale LLMs (smolLM-1.7B, phi3-mini-4k, mistral-7B) show up to 30% storage savings over uint8 and 65% over uint4 models, with 31.9-146.6% faster inference on memory-limited devices like the NVIDIA JETSON P3450. EntroLLM requires no retraining and is compatible with existing post-training quantization pipelines, making it practical for edge LLM deployment.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 26 of its 34 references that are in the map. First posted on arXiv 5 May 2025.

Builds on

26 of 34 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttention12 Sep

Other references · 25

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  5. 2023Mistral 7Bcited by 3,065
  6. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  7. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  8. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
  9. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
  10. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
  11. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 643
  12. 2022Efficiently Scaling Transformer Inferencecited by 529
13 more
  1. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  2. 2021A White Paper on Neural Network Quantizationcited by 453
  3. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 413
  4. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 359
  5. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 285
  6. 2024A Survey on Efficient Inference for Large Language Modelscited by 197
  7. 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 176
  8. 2021Understanding and Overcoming the Challenges of Efficient Transformer Quantizationcited by 139
  9. 2024AI and Memory Wallcited by 89
  10. 2023QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Modelscited by 42
  11. 2024TernaryLLM: Ternarized Large Language Modelcited by 15
  12. 2018Deep Positron: A Deep Neural Network Using the Posit Number Systemcited by 13
  13. 2019Neural Network Training with Approximate Logarithmic Computationscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Chameleon: Dynamic Format Adapter for Efficient Diffusioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Huff-LLM: End-to-End Lossless Compression for Efficient LLM Inference
  2. 2026ZipServ: Fast and Memory-Efficient LLM Inference with Hardware-Aware Lossless Compression
  3. 2026On the Expressive Power of Weight Quantization in Large Language Models
  4. 2023Compressing LLMs: The Truth is Rarely Pure and Never Simple
  5. 2025Optimizing LLMs for Resource-Constrained Environments: A Survey of Model Compression Techniques
  6. 2024ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models
  7. 2024KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization
  8. 2023Just CHOP: Embarrassingly Simple LLM Compression