# Lossless data compression by large models

arXiv: https://arxiv.org/abs/2407.07723 · HTML: https://mlsanity.com/paper/2407.07723/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 51 references that are in the map.
- First posted on arXiv 24 Jun 2024.

## Builds on (most cited first, 24 of 24)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation](https://mlsanity.com/paper/2101.00390/) (2021)
- [ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Coding](https://mlsanity.com/paper/2203.10886/) (2022)
- [Language Modeling Is Compression](https://mlsanity.com/paper/2309.10668/) (2023)
- [Deep Contextual Video Compression](https://mlsanity.com/paper/2109.15047/) (2021)
- [Lossy Image Compression with Conditional Diffusion Models](https://mlsanity.com/paper/2209.06950/) (2022)
- [Hybrid Spatial-Temporal Entropy Modelling for Neural Video Compression](https://mlsanity.com/paper/2207.05894/) (2022)
- [Neural Video Compression with Feature Modulation](https://mlsanity.com/paper/2402.17414/) (2024)
- [Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Dataset](https://mlsanity.com/paper/2207.00220/) (2022)
- [Meta-learning of Sequential Strategies](https://mlsanity.com/paper/1905.03030/) (2019)
- [Generative Compression](https://mlsanity.com/paper/1703.01467/) (2017)
- [Lossy Image Compression with Foundation Diffusion Models](https://mlsanity.com/paper/2404.08580/) (2024)
- [Learning Universal Predictors](https://mlsanity.com/paper/2401.14953/) (2024)
- [Idempotence and Perceptual Image Compression](https://mlsanity.com/paper/2401.08920/) (2024)
- [Beyond Language Models: Byte Models are Digital World Simulators](https://mlsanity.com/paper/2402.19155/) (2024)
- [MeDAL: Medical Abbreviation Disambiguation Dataset for Natural Language Understanding Pretraining](https://mlsanity.com/paper/2012.13978/) (2020)
- [LC-FDNet: Learned Lossless Image Compression with Frequency Decomposition Network](https://mlsanity.com/paper/2112.06417/) (2021)
- [Approximating Human-Like Few-shot Learning with GPT-based Compression](https://mlsanity.com/paper/2308.06942/) (2023)
- [A Theory of Human-Like Few-Shot Learning](https://mlsanity.com/paper/2301.01047/) (2023)

## Built on by (most cited first, 20 of 20)

- [Mechanistic Indicators of Understanding in Large Language Models](https://mlsanity.com/paper/2507.08017/) (2025)
- [Neural Entropy](https://mlsanity.com/paper/2409.03817/) (2024)
- [Information Capacity: Evaluating the Efficiency of Large Language Models via Text Compression](https://mlsanity.com/paper/2511.08066/) (2025)
- [Revisiting MLLM Token Technology through the Lens of Classical Visual Coding](https://mlsanity.com/paper/2508.13460/) (2025)
- [SelfAI: A self-directed framework for long-horizon scientific discovery](https://mlsanity.com/paper/2512.00403/) (2025)
- [Compression Tells Intelligence: Visual Coding, Visual Token Technology, and the Unification](https://mlsanity.com/paper/2601.20742/) (2026)
- [AgentGC: Evolutionary Learning-based Lossless Compression for Genomics Data with LLM-driven Multiple Agent](https://mlsanity.com/paper/2601.13559/) (2026)
- [Grokking From Abstraction to Intelligence](https://mlsanity.com/paper/2603.29262/) (2026)
- [Adapting Diffusion Language Models for Lossless Pixel-Level Image Transmission](https://mlsanity.com/paper/2606.06273/) (2026)
- [Way to Build Native AI-driven 6G Air Interface: Principles, Roadmap, and Outlook](https://mlsanity.com/paper/2508.15277/) (2025)
- [The Algorithmic Regulator](https://mlsanity.com/paper/2510.10300/) (2025)
- [Shakespeare, Entropy and Educated Monkeys](https://mlsanity.com/paper/2512.11880/) (2025)
- [Benchmarking Language Modeling for Lossless Compression of Full-Fidelity Audio](https://mlsanity.com/paper/2603.08683/) (2026)
- [NeuralLVC: Neural Lossless Video Compression via Masked Diffusion with Temporal Conditioning](https://mlsanity.com/paper/2604.03353/) (2026)
- [Thinking as Compression: Your Reasoning Model is Secretly a Context Compressor](https://mlsanity.com/paper/2605.28713/) (2026)
- [Text Distance from Nested and Hierarchical Repetitions: A Compression-Based Perspective](https://mlsanity.com/paper/2607.05416/) (2026)
- [LUMI: Tokenizer-Agnostic LLM-Based Lossless Image Compression](https://mlsanity.com/paper/2607.08221/) (2026)
- [Beyond Representational Similarity: Source-Conditioned Description-Length Gain for Generative Plagiarism Detection and Candidate Source Reranking](https://mlsanity.com/paper/2608.03859/) (2026)
- [FLM: Frequency-Aware Language Models for Generative Image Compression](https://mlsanity.com/paper/2608.28687/) (2026)
- [HyperZip: Efficient Data Compression through Personalized Diffusion LLMs with Hypernetworks](https://mlsanity.com/paper/2609.36357/) (2026)

## Abstract (arXiv)

Modern data compression methods are slowly reaching their limits after 80 years of research, millions of papers, and wide range of applications. Yet, the extravagant 6G communication speed requirement raises a major open question for revolutionary new ideas of data compression. We have previously shown all understanding or learning are compression, under reasonable assumptions. Large language models (LLMs) understand data better than ever before. Can they help us to compress data? The LLMs may be seen to approximate the uncomputable Solomonoff induction. Therefore, under this new uncomputable paradigm, we present LMCompress. LMCompress shatters all previous lossless compression algorithms, doubling the lossless compression ratios of JPEG-XL for images, FLAC for audios, and H.264 for videos, and quadrupling the compression ratio of bz2 for texts. The better a large model understands the data, the better LMCompress compresses.