# An Introduction to Compression-Based Machine Learning

arXiv: https://arxiv.org/abs/2609.21309 · HTML: https://mlsanity.com/paper/2609.21309/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 65 references that are in the map.
- First posted on arXiv 18 Sep 2026.

## Builds on (most cited first, 17 of 17)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Conditional Image Generation with PixelCNN Decoders](https://mlsanity.com/paper/1606.05328/) (2016)
- [Language Modeling Is Compression](https://mlsanity.com/paper/2309.10668/) (2023)
- [Practical Full Resolution Learned Lossless Image Compression](https://mlsanity.com/paper/1811.12817/) (2018)
- [An Introduction to Neural Data Compression](https://mlsanity.com/paper/2202.06533/) (2022)
- [LLMZip: Lossless Text Compression using Large Language Models](https://mlsanity.com/paper/2306.04050/) (2023)
- [A Survey of Machine Learning Methods and Challenges for Windows Malware Classification](https://mlsanity.com/paper/2006.09271/) (2020)
- [A New Burrows Wheeler Transform Markov Distance](https://mlsanity.com/paper/1912.13046/) (2019)
- [CALLIC: Content Adaptive Learning for Lossless Image Compression](https://mlsanity.com/paper/2412.17464/) (2024)
- [On Normalized Compression Distance and Large Malware](https://mlsanity.com/paper/1509.00689/) (2015)
- [Few-Shot Non-Parametric Learning with Deep Latent Variable Model](https://mlsanity.com/paper/2206.11573/) (2022)
- [Text Ranking and Classification using Data Compression](https://mlsanity.com/paper/2109.11577/) (2021)
- [Neural Normalized Compression Distance and the Disconnect Between Compression and Classification](https://mlsanity.com/paper/2410.15280/) (2024)
- [Detecting Malware with Information Complexity](https://mlsanity.com/paper/1502.07661/) (2015)
- [Cybersecurity is the True Frontier for Generative AI Success or Failure](https://mlsanity.com/paper/2606.28929/) (2026)
- [Lightweight Conceptual Dictionary Learning for Text Classification Using Information Compression](https://mlsanity.com/paper/2405.01584/) (2024)

## Abstract (arXiv)

Any lossless compression algorithm (like gzip) may be converted into a machine learning method, via either Normalized Compression Distance or the Minimum Description Length principle. Any auto-regressive model may be converted into a lossless compression method via entropy coding. This seemingly circular dependence has unrealized potential in modern artificial intelligence and machine learning, and we survey and formalize the various strategies that have been used to leverage compression for machine learning. We introduce and empirically validate a design framework for compression-based ML, finding compression-based methods competitive with conventional baselines and decisively stronger on malware. We find that varying these design choices yields accuracy gains of up to 0.62.