# The Design, Implementation, and Deployment of a System to Transparently Compress Hundreds of Petabytes of Image Files for a File-Storage Service

arXiv: https://arxiv.org/abs/1704.06192 · HTML: https://mlsanity.com/paper/1704.06192/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Mar 2017.

## Built on by (most cited first, 6 of 6)

- [Practical Learned Lossless JPEG Recompression with Multi-Level Cross-Channel Entropy Model in the DCT Domain](https://mlsanity.com/paper/2203.16357/) (2022)
- [DeepSketch: A New Machine Learning-Based Reference Search Technique for Post-Deduplication Delta Compression](https://mlsanity.com/paper/2202.10584/) (2022)
- [Learned Lossless Compression for JPEG via Frequency-Domain Prediction](https://mlsanity.com/paper/2303.02666/) (2023)
- [Learned Lossless JPEG Transcoding via Joint Lossy and Residual Compression](https://mlsanity.com/paper/2208.11673/) (2022)
- [Unified learning-based lossy and lossless JPEG recompression](https://mlsanity.com/paper/2312.02705/) (2023)
- [FLM: Frequency-Aware Language Models for Generative Image Compression](https://mlsanity.com/paper/2608.28687/) (2026)

## Abstract (arXiv)

We report the design, implementation, and deployment of Lepton, a fault-tolerant system that losslessly compresses JPEG images to 77% of their original size on average. Lepton replaces the lowest layer of baseline JPEG compression-a Huffman code-with a parallelized arithmetic code, so that the exact bytes of the original JPEG file can be recovered quickly. Lepton matches the compression efficiency of the best prior work, while decoding more than nine times faster and in a streaming manner. Lepton has been released as open-source software and has been deployed for a year on the Dropbox file-storage backend. As of February 2017, it had compressed more than 203 PiB of user JPEG files, saving more than 46 PiB.