# ULPack: Reference-Conditioned Lossless Compression of Checkpoint Trajectories

arXiv: https://arxiv.org/abs/2610.14184 · HTML: https://mlsanity.com/paper/2610.14184/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 19 references that are in the map.
- First posted on arXiv 9 Oct 2026.

## Builds on (most cited first, 13 of 13)

- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [OLMo: Accelerating the Science of Language Models](https://mlsanity.com/paper/2402.00838/) (2024)
- [LLM360: Towards Fully Transparent Open-Source LLMs](https://mlsanity.com/paper/2312.06550/) (2023)
- [The MultiBERTs: BERT Reproductions for Robustness Analysis](https://mlsanity.com/paper/2106.16163/) (2021)
- [Model Zoos: A Dataset of Diverse Populations of Neural Network Models](https://mlsanity.com/paper/2209.14764/) (2022)
- [ZipNN: Lossless Compression for AI Models](https://mlsanity.com/paper/2411.05239/) (2024)
- [On Efficient Constructions of Checkpoints](https://mlsanity.com/paper/2009.13003/) (2020)
- [ExCP: Extreme LLM Checkpoint Compression via Weight-Momentum Joint Shrinking](https://mlsanity.com/paper/2406.11257/) (2024)
- [Lossless Compression for LLM Tensor Incremental Snapshots](https://mlsanity.com/paper/2505.09810/) (2025)
- [An Efficient Compression of Deep Neural Network Checkpoints Based on Prediction and Context Modeling](https://mlsanity.com/paper/2506.12000/) (2025)
- [VIREL: Route-Local Lattice Residual Compression for Exact and Error-Bounded Floating-Point Time Series](https://mlsanity.com/paper/2607.22433/) (2026)
- [Lossless Tensor Compression as Program Synthesis](https://mlsanity.com/paper/2608.02162/) (2026)

## Abstract (arXiv)

Studying model training requires checkpoint trajectories, whose storage cost limits temporal resolution. Although adjacent checkpoints are numerically close, existing lossless codecs either operate on XOR bit patterns or ignore trajectory-specific conditional structure. We show that a reference weight's exponent predicts the bit length of its displacement in ordered floating-point space without side information. We introduce ULPack, a lossless codec which exploits this dependence by conditioning entropy models of ordered-integer residuals on the reference exponent. Across bfloat16, float16, and float32 trajectories, ULPack reduces storage by up to 27% against ZipNN, 21% against zstd-19, and 11.5% against FM-Delta. Across two dense trajectories, compression improves as updates shrink and reference conditioning strengthens, while near uniform low-order bits set the remaining compression floor and earlier checkpoints add little information.