# A Model-Driven Lossless Compression Algorithm Resistant to Mismatch

arXiv: https://arxiv.org/abs/2601.17684 · HTML: https://mlsanity.com/paper/2601.17684/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 41 references that are in the map.
- First posted on arXiv 25 Jan 2026.

## Builds on (most cited first, 20 of 20)

- [Language Modeling Is Compression](https://mlsanity.com/paper/2309.10668/) (2023)
- [Full Resolution Image Compression with Recurrent Neural Networks](https://mlsanity.com/paper/1608.05148/) (2016)
- [Interactive Supercomputing on 40,000 Cores for Machine Learning and Data Analysis](https://mlsanity.com/paper/1807.07814/) (2018)
- [Practical Full Resolution Learned Lossless Image Compression](https://mlsanity.com/paper/1811.12817/) (2018)
- [Non-Determinism of "Deterministic" LLM Settings](https://mlsanity.com/paper/2408.04667/) (2024)
- [LLMZip: Lossless Text Compression using Large Language Models](https://mlsanity.com/paper/2306.04050/) (2023)
- [Reproducibility in Machine Learning-based Research: Overview, Barriers and Drivers](https://mlsanity.com/paper/2406.14325/) (2024)
- [DeepZip: Lossless Data Compression using Recurrent Neural Networks](https://mlsanity.com/paper/1811.08162/) (2018)
- [A Fast Transformer-based General-Purpose Lossless Compressor](https://mlsanity.com/paper/2203.16114/) (2022)
- [Impacts of floating-point non-associativity on reproducibility for HPC and deep learning applications](https://mlsanity.com/paper/2408.05148/) (2024)
- [Towards Training Reproducible Deep Learning Models](https://mlsanity.com/paper/2202.02326/) (2022)
- [FineZip : Pushing the Limits of Large Language Models for Practical Lossless Text Compression](https://mlsanity.com/paper/2409.17141/) (2024)
- [LC-FDNet: Learned Lossless Image Compression with Frequency Decomposition Network](https://mlsanity.com/paper/2112.06417/) (2021)
- [Non-Determinism in TensorFlow ResNets](https://mlsanity.com/paper/2001.11396/) (2020)
- [Large Language Models for Lossless Image Compression: Next-Pixel Prediction in Language Space is All You Need](https://mlsanity.com/paper/2411.12448/) (2024)
- [Non-Determinism and the Lawlessness of Machine Learning Code](https://mlsanity.com/paper/2206.11834/) (2022)
- [Syntactically Informed Text Compression with Recurrent Neural Networks](https://mlsanity.com/paper/1608.02893/) (2016)
- [Examining the Effect of Implementation Factors on Deep Learning Reproducibility](https://mlsanity.com/paper/2312.06633/) (2023)
- [Investigating the Impact of Randomness on Reproducibility in Computer Vision: A Study on Applications in Civil Engineering and Medicine](https://mlsanity.com/paper/2410.02806/) (2024)
- [An Efficient Compression of Deep Neural Network Checkpoints Based on Prediction and Context Modeling](https://mlsanity.com/paper/2506.12000/) (2025)

## Abstract (arXiv)

Due to the fundamental connection between next-symbol prediction and compression, modern predictive models, such as large language models (LLMs), can be combined with entropy coding to achieve compression rates that surpass those of standard compression algorithms. However, this approach relies on the assumption that the predictive model produces identical output distributions at both the encoder and decoder, since even small mismatches can cause the decoding to fail. This assumption often fails with complex predictive models, particularly those based on neural networks, a phenomenon referred to as non-determinism. In this work, we propose a new compression algorithm based on next-token prediction that is robust to arbitrarily large, but structured, prediction mismatches. We prove the correctness of the proposed scheme under a formal mismatch certification, characterize its theoretical performance, and validate it experimentally on real datasets. Our results demonstrate reliable operation within the certified mismatch regime while achieving compression ratios that exceed those of commonly used compression methods.