# FineZip : Pushing the Limits of Large Language Models for Practical Lossless Text Compression

arXiv: https://arxiv.org/abs/2409.17141 · HTML: https://mlsanity.com/paper/2409.17141/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 16 references that are in the map.
- First posted on arXiv 25 Sep 2024.

## Builds on (most cited first, 5 of 5)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [DeepZip: Lossless Data Compression using Recurrent Neural Networks](https://mlsanity.com/paper/1811.08162/) (2018)

## Built on by (most cited first, 18 of 18)

- [Cramming 1568 Tokens into a Single Vector and Back Again: Exploring the Limits of Embedding Space Capacity](https://mlsanity.com/paper/2502.13063/) (2025)
- [Compression via Pre-trained Transformers: A Study on Byte-Level Multimodal Data](https://mlsanity.com/paper/2410.05078/) (2024)
- [Text summarization via global structure awareness](https://mlsanity.com/paper/2602.09821/) (2026)
- [Information Capacity: Evaluating the Efficiency of Large Language Models via Text Compression](https://mlsanity.com/paper/2511.08066/) (2025)
- [Lossless Compression of Large Language Model-Generated Text via Next-Token Prediction](https://mlsanity.com/paper/2505.06297/) (2025)
- [Nacrith: Neural Lossless Compression via Ensemble Context Modeling and High-Precision CDF Coding](https://mlsanity.com/paper/2602.19626/) (2026)
- [Test-Time Steering for Lossless Text Compression via Weighted Product of Experts](https://mlsanity.com/paper/2511.10660/) (2025)
- [DualComp: End-to-End Learning of a Unified Dual-Modality Lossless Compressor](https://mlsanity.com/paper/2505.16256/) (2025)
- [OmniZip: Learning a Unified and Lightweight Lossless Compressor for Multi-Modal Data](https://mlsanity.com/paper/2602.22286/) (2026)
- [In-Context Source and Channel Coding](https://mlsanity.com/paper/2601.10267/) (2026)
- [LLM-based Source Code Compression via Thresholded Symbol Ranking](https://mlsanity.com/paper/2607.24192/) (2026)
- [Information Abundance Paradox: Long-Context Training Undermines Parametric Knowledge](https://mlsanity.com/paper/2608.12218/) (2026)
- [A Model-Driven Lossless Compression Algorithm Resistant to Mismatch](https://mlsanity.com/paper/2601.17684/) (2026)
- [Micro-Diffusion Compression - Binary Tree Tweedie Denoising for Online Probability Estimation](https://mlsanity.com/paper/2603.08771/) (2026)
- [StateSMix: Online Lossless Compression via Mamba State Space Models and Sparse N-gram Context Mixing](https://mlsanity.com/paper/2605.02904/) (2026)
- [Diffuse to Compress: Leveraging Diffusion LMs for Lossless Compression](https://mlsanity.com/paper/2608.11249/) (2026)
- [Cadence: Error-Bounded Lossy Compression of Demand Time Series with a Time-Series Foundation Model](https://mlsanity.com/paper/2609.06008/) (2026)
- [Lossy Compressive Text Autoencoders](https://mlsanity.com/paper/2610.10738/) (2026)

## Abstract (arXiv)

While the language modeling objective has been shown to be deeply connected with compression, it is surprising that modern LLMs are not employed in practical text compression systems. In this paper, we provide an in-depth analysis of neural network and transformer-based compression techniques to answer this question. We compare traditional text compression systems with neural network and LLM-based text compression methods. Although LLM-based systems significantly outperform conventional compression methods, they are highly impractical. Specifically, LLMZip, a recent text compression system using Llama3-8B requires 9.5 days to compress just 10 MB of text, although with huge improvements in compression ratios. To overcome this, we present FineZip - a novel LLM-based text compression system that combines ideas of online memorization and dynamic context to reduce the compression time immensely. FineZip can compress the above corpus in approximately 4 hours compared to 9.5 days, a 54 times improvement over LLMZip and comparable performance. FineZip outperforms traditional algorithmic compression methods with a large margin, improving compression ratios by approximately 50%. With this work, we take the first step towards making lossless text compression with LLMs a reality. While FineZip presents a significant step in that direction, LLMs are still not a viable solution for large-scale text compression. We hope our work paves the way for future research and innovation to solve this problem.