# AlphaZip: Neural Network-Enhanced Lossless Text Compression

arXiv: https://arxiv.org/abs/2409.15046 · HTML: https://mlsanity.com/paper/2409.15046/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 26 references that are in the map.
- First posted on arXiv 23 Sep 2024.

## Builds on (most cited first, 7 of 7)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Generative Pre-trained Transformer: A Comprehensive Review on Enabling Technologies, Potential Applications, Emerging Challenges, and Future Directions](https://mlsanity.com/paper/2305.10435/) (2023)
- [LLMZip: Lossless Text Compression using Large Language Models](https://mlsanity.com/paper/2306.04050/) (2023)
- [DeepZip: Lossless Data Compression using Recurrent Neural Networks](https://mlsanity.com/paper/1811.08162/) (2018)
- [Complexity of Word Collocation Networks: A Preliminary Structural Analysis](https://mlsanity.com/paper/1310.5111/) (2013)
- [PEFT-Ref: A Modular Reference Architecture and Typology for Parameter-Efficient Finetuning Techniques](https://mlsanity.com/paper/2304.12410/) (2023)

## Built on by (most cited first, 7 of 7)

- [Information Capacity: Evaluating the Efficiency of Large Language Models via Text Compression](https://mlsanity.com/paper/2511.08066/) (2025)
- [Test-Time Steering for Lossless Text Compression via Weighted Product of Experts](https://mlsanity.com/paper/2511.10660/) (2025)
- [In-Context Source and Channel Coding](https://mlsanity.com/paper/2601.10267/) (2026)
- [LLM-based Source Code Compression via Thresholded Symbol Ranking](https://mlsanity.com/paper/2607.24192/) (2026)
- [GPT as a Monte Carlo Language Tree: A Probabilistic Perspective](https://mlsanity.com/paper/2501.07641/) (2025)
- [Bayesian Network Fusion of Large Language Models for Sentiment Analysis](https://mlsanity.com/paper/2510.26484/) (2025)
- [Diffuse to Compress: Leveraging Diffusion LMs for Lossless Compression](https://mlsanity.com/paper/2608.11249/) (2026)

## Abstract (arXiv)

Data compression continues to evolve, with traditional information theory methods being widely used for compressing text, images, and videos. Recently, there has been growing interest in leveraging Generative AI for predictive compression techniques. This paper introduces a lossless text compression approach using a Large Language Model (LLM). The method involves two key steps: first, prediction using a dense neural network architecture, such as a transformer block; second, compressing the predicted ranks with standard compression algorithms like Adaptive Huffman, LZ77, or Gzip. Extensive analysis and benchmarking against conventional information-theoretic baselines demonstrate that neural compression offers improved performance.