# On Efficient Constructions of Checkpoints

arXiv: https://arxiv.org/abs/2009.13003 · HTML: https://mlsanity.com/paper/2009.13003/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Sep 2020.

## Built on by (most cited first, 7 of 7)

- [ExCP: Extreme LLM Checkpoint Compression via Weight-Momentum Joint Shrinking](https://mlsanity.com/paper/2406.11257/) (2024)
- [I/O in Machine Learning Applications on HPC Systems: A 360-degree Survey](https://mlsanity.com/paper/2404.10386/) (2024)
- [Inshrinkerator: Compressing Deep Learning Training Checkpoints via Dynamic Quantization](https://mlsanity.com/paper/2306.11800/) (2023)
- [An Efficient Compression of Deep Neural Network Checkpoints Based on Prediction and Context Modeling](https://mlsanity.com/paper/2506.12000/) (2025)
- [ECRM: Efficient Fault Tolerance for Recommendation Model Training via Erasure Coding](https://mlsanity.com/paper/2104.01981/) (2021)
- [BitSnap: Checkpoint Sparsification and Quantization in LLM Training](https://mlsanity.com/paper/2511.12376/) (2025)
- [ULPack: Reference-Conditioned Lossless Compression of Checkpoint Trajectories](https://mlsanity.com/paper/2610.14184/) (2026)

## Abstract (arXiv)

Efficient construction of checkpoints/snapshots is a critical tool for training and diagnosing deep learning models. In this paper, we propose a lossy compression scheme for checkpoint constructions (called LC-Checkpoint). LC-Checkpoint simultaneously maximizes the compression rate and optimizes the recovery speed, under the assumption that SGD is used to train the model. LC-Checkpointuses quantization and priority promotion to store the most crucial information for SGD to recover, and then uses a Huffman coding to leverage the non-uniform distribution of the gradient scales. Our extensive experiments show that LC-Checkpoint achieves a compression rate up to 28× and recovery speedup up to 5.77× over a state-of-the-art algorithm (SCAR).