# Progressive Compressed Records: Taking a Byte out of Deep Learning Data

arXiv: https://arxiv.org/abs/1911.00472 · HTML: https://mlsanity.com/paper/1911.00472/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Nov 2019.

## Built on by (most cited first, 4 of 4)

- [Plumber: Diagnosing and Removing Performance Bottlenecks in Machine Learning Data Pipelines](https://mlsanity.com/paper/2111.04131/) (2021)
- [I/O in Machine Learning Applications on HPC Systems: A 360-degree Survey](https://mlsanity.com/paper/2404.10386/) (2024)
- [MinatoLoader: Accelerating Machine Learning Training Through Efficient Data Preprocessing](https://mlsanity.com/paper/2509.10712/) (2025)
- [AutoMC: Automated Model Compression based on Domain Knowledge and Progressive search strategy](https://mlsanity.com/paper/2201.09884/) (2022)

## Abstract (arXiv)

Deep learning accelerators efficiently train over vast and growing amounts of data, placing a newfound burden on commodity networks and storage devices. A common approach to conserve bandwidth involves resizing or compressing data prior to training. We introduce Progressive Compressed Records (PCRs), a data format that uses compression to reduce the overhead of fetching and transporting data, effectively reducing the training time required to achieve a target accuracy. PCRs deviate from previous storage formats by combining progressive compression with an efficient storage layout to view a single dataset at multiple fidelities---all without adding to the total dataset size. We implement PCRs and evaluate them on a range of datasets, training tasks, and hardware architectures. Our work shows that: (i) the amount of compression a dataset can tolerate exceeds 50% of the original encoding for many DL training tasks; (ii) it is possible to automatically and efficiently select appropriate compression levels for a given task; and (iii) PCRs enable tasks to readily access compressed data at runtime---utilizing as little as half the training bandwidth and thus potentially doubling training speed.