mlsanityData to 7 Oct 2026

LLMZip: Lossless Text Compression using Large Language Models

arXiv 2306.04050 ↗First posted 6 Jun 2023cs.IT cs.CL cs.LGcited by 44

Not on the map. Why?

Abstract (arXiv)

We provide new estimates of an asymptotic upper bound on the entropy of English using the large language model LLaMA-7B as a predictor for the next token given a window of past tokens. This estimate is significantly smaller than currently available estimates in citecover1978convergent, citelutati2023focus. A natural byproduct is an algorithm for lossless compression of English text which combines the prediction from the large language model with a lossless compression scheme. Preliminary results from limited experiments suggest that our scheme outperforms state-of-the-art text compression schemes such as BSC, ZPAQ, and paq8h.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Builds on 4 of its 11 references that are in the map. First posted on arXiv 6 Jun 2023.

Builds on

4 of 11 references.

References · 4

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  3. 2018DeepZip: Lossless Data Compression using Recurrent Neural Networkscited by 32
  4. 2023Focus Your Attention (with Adaptive IIR Filters)cited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 14; 2025: 10; 2026: 17.

44 papers.

Sort:

By area:

  1. 2023Language Modeling Is Compressioncited by 225
  2. 2024A Survey of Resource-efficient LLM and Multimodal Foundation Modelscited by 84
  3. 2023EdgeMoE: Empowering Sparse Large Language Models on Mobile Devicescited by 65
  4. 2023SiDA-MoE: Sparsity-Inspired Data-Aware Serving for Efficient and Scalable Large Mixture-of-Experts Modelscited by 36
  5. 2024Diff-eRank: A Novel Rank-Based Metric for Evaluating Large Language Modelscited by 35
  6. 2024Training LLMs over Neurally Compressed Textcited by 16
  7. 2024Compression via Pre-trained Transformers: A Study on Byte-Level Multimodal Datacited by 13
  8. 2024SWIFT: Semantic Watermarking for Image Forgery Thwartingcited by 8
  9. 2024Towards Optimal Learning of Language Modelscited by 7
  10. 2024AlphaZip: Neural Network-Enhanced Lossless Text Compressioncited by 7
  11. 2024In-context learning and Occam's razorcited by 7
  12. 2025Exploiting Vocabulary Frequency Imbalance in Language Model Pre-trainingcited by 7
32 more
  1. 2024Evaluating Large Language Models for Generalization and Robustness via Data Compressioncited by 5
  2. 2024The Information of Large Language Model Geometrycited by 5
  3. 2025The KoLMogorov Test: Compression by Code Generationcited by 5
  4. 2025Joint Lossless Compression and Steganography for Medical Images via Large Language Modelscited by 5
  5. 2024LMM-driven Semantic Image-Text Coding for Ultra Low-bitrate Learned Image Compressioncited by 4
  6. 2026Nacrith: Neural Lossless Compression via Ensemble Context Modeling and High-Precision CDF Codingcited by 4
  7. 2025Fast, Secure, and High-Capacity Image Watermarking with Autoencoded Text Vectorscited by 3
  8. 2024Large Language Model Evaluation via Matrix Nuclear-Normcited by 2
  9. 2024An Enhanced Text Compression Approach Using Transformer-based Language Modelscited by 2
  10. 2025DualComp: End-to-End Learning of a Unified Dual-Modality Lossless Compressorcited by 2
  11. 2025Prediction-Powered Communication with Distortion Guaranteescited by 2
  12. 2026Revisiting Data Compression with Language Modelingcited by 2
  13. 2026OmniZip: Learning a Unified and Lightweight Lossless Compressor for Multi-Modal Datacited by 2
  14. 2024Coding for Intelligence from the Perspective of Categorycited by 1
  15. 2025The Lossy Horizon: Error-Bounded Predictive Coding for Lossy Text Compression (Episode I)cited by 1
  16. 2025A high-capacity linguistic steganography based on entropy-driven rank-token mappingcited by 1
  17. 2026In-Context Source and Channel Codingcited by 1
  18. 2026Semantic Chunking and the Entropy of Natural Languagecited by 1
  19. 2026From Similarity to Structure: Training-free LLM Context Compression with Hybrid Graph Priorscited by 1
  20. 2026Adapting Diffusion Language Models for Lossless Pixel-Level Image Transmissioncited by 1
  21. 2026LLM-based Source Code Compression via Thresholded Symbol Rankingcited by 1
  22. 2026Information Abundance Paradox: Long-Context Training Undermines Parametric Knowledgecited by 1
  23. 2025GPT as a Monte Carlo Language Tree: A Probabilistic Perspectivecited by 0
  24. 2025The Algorithmic Regulatorcited by 0
  25. 2026A Model-Driven Lossless Compression Algorithm Resistant to Mismatchcited by 0
  26. 2026SSR: A Generic Framework for Text-Aided Map Compression for Localizationcited by 0
  27. 2026Real-Time Text Transmission via LLM-Based Entropy Coding over Fixed-Rate Channelscited by 0
  28. 2026StateSMix: Online Lossless Compression via Mamba State Space Models and Sparse N-gram Context Mixingcited by 0
  29. 2026The Value of a Prompt: An LLM-Relative Kolmogorov-Complexity Approachcited by 0
  30. 2026An Introduction to Compression-Based Machine Learningcited by 0
  31. 2026Semantic Watermarking for Malicious Image Manipulation Detectioncited by 0
  32. 2026Lossy Compressive Text Autoencoderscited by 0