# Morphing-based Compression for Data-centric ML Pipelines

arXiv: https://arxiv.org/abs/2504.11067 · HTML: https://mlsanity.com/paper/2504.11067/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 121 references that are in the map.
- First posted on arXiv 15 Apr 2025.

## Builds on (most cited first, 26 of 26)

- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations](https://mlsanity.com/paper/1609.07061/) (2016)
- [Snorkel: Rapid Training Data Creation with Weak Supervision](https://mlsanity.com/paper/1711.10160/) (2017)
- [LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks](https://mlsanity.com/paper/1807.10029/) (2018)
- [Data Programming: Creating Large Training Sets, Quickly](https://mlsanity.com/paper/1605.07723/) (2016)
- [A Study of BFLOAT16 for Deep Learning Training](https://mlsanity.com/paper/1905.12322/) (2019)
- [Advances in Pre-Training Distributed Word Representations](https://mlsanity.com/paper/1712.09405/) (2017)
- [Training Deep Neural Networks with 8-bit Floating Point Numbers](https://mlsanity.com/paper/1812.08011/) (2018)
- [Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Data](https://mlsanity.com/paper/1912.07768/) (2019)
- [MLIR: A Compiler Infrastructure for the End of Moore's Law](https://mlsanity.com/paper/2002.11054/) (2020)
- [DeepHoyer: Learning Sparser Neural Network with Differentiable Scale-Invariant Sparsity Measures](https://mlsanity.com/paper/1908.09979/) (2019)
- [Flexpoint: An Adaptive Numerical Format for Efficient Training of Deep Neural Networks](https://mlsanity.com/paper/1711.02213/) (2017)
- [SystemDS: A Declarative Machine Learning System for the End-to-End Data Science Lifecycle](https://mlsanity.com/paper/1909.02976/) (2019)
- [Learning Models over Relational Data using Sparse Tensors and Functional Dependencies](https://mlsanity.com/paper/1703.04780/) (2017)
- [The Relational Data Borg is Learning](https://mlsanity.com/paper/2008.07864/) (2020)
- [BlinkML: Efficient Maximum Likelihood Estimation with Probabilistic Guarantees](https://mlsanity.com/paper/1812.10564/) (2018)
- [Improving Matrix-vector Multiplication via Lossless Grammar-Compressed Matrices](https://mlsanity.com/paper/2203.14540/) (2022)
- [UniSparse: An Intermediate Language for General Sparse Format Customization](https://mlsanity.com/paper/2403.05802/) (2024)
- [Compilation of Modular and General Sparse Workspaces](https://mlsanity.com/paper/2404.04541/) (2024)
- [Tuple-oriented Compression for Large-scale Mini-batch Stochastic Gradient Descent](https://mlsanity.com/paper/1702.06943/) (2017)
- [High-Ratio Compression for Machine-Generated Data](https://mlsanity.com/paper/2311.13947/) (2023)
- [MorphStore: Analytical Query Engine with a Holistic Compression-Enabled Processing Model](https://mlsanity.com/paper/2004.09350/) (2020)
- [Impossibility Results for Grammar-Compressed Linear Algebra](https://mlsanity.com/paper/2010.14181/) (2020)
- [Number Parsing at a Gigabyte per Second](https://mlsanity.com/paper/2101.11408/) (2021)
- [LEA: A Learned Encoding Advisor for Column Stores](https://mlsanity.com/paper/2105.08830/) (2021)
- [Grammar Boosting: A New Technique for Proving Lower Bounds for Computation over Compressed Data](https://mlsanity.com/paper/2307.08833/) (2023)

## Abstract (arXiv)

Data-centric ML pipelines extend traditional machine learning (ML) pipelines -- of feature transformations and ML model training -- by outer loops for data cleaning, augmentation, and feature engineering to create high-quality input data. Existing lossless matrix compression applies lightweight compression schemes to numeric matrices and performs linear algebra operations such as matrix-vector multiplications directly on the compressed representation but struggles to efficiently rediscover structural data redundancy. Compressed operations are effective at fitting data in available memory, reducing I/O across the storage-memory-cache hierarchy, and improving instruction parallelism. The applied data cleaning, augmentation, and feature transformations provide a rich source of information about data characteristics such as distinct items, column sparsity, and column correlations. In this paper, we introduce BWARE -- an extension of AWARE for workload-aware lossless matrix compression -- that pushes compression through feature transformations and engineering to leverage information about structural transformations. Besides compressed feature transformations, we introduce a novel technique for lightweight morphing of a compressed representation into workload-optimized compressed representations without decompression. BWARE shows substantial end-to-end runtime improvements, reducing the execution time for training data-centric ML pipelines from days to hours.