# SZ3: A Modular Framework for Composing Prediction-Based Error-Bounded Lossy Compressors

arXiv: https://arxiv.org/abs/2111.02925 · HTML: https://mlsanity.com/paper/2111.02925/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Nov 2021.

## Built on by (most cited first, 30 of 33)

- [Compressing multidimensional weather and climate data into neural networks](https://mlsanity.com/paper/2210.12538/) (2022)
- [Advances in APPFL: A Comprehensive and Extensible Federated Learning Framework](https://mlsanity.com/paper/2409.11585/) (2024)
- [ECNR: Efficient Compressive Neural Representation of Time-Varying Volumetric Datasets](https://mlsanity.com/paper/2311.12831/) (2023)
- [Adaptively Placed Multi-Grid Scene Representation Networks for Large-Scale Data Visualization](https://mlsanity.com/paper/2308.02494/) (2023)
- [Lossy Neural Compression for Geospatial Analytics: A Review](https://mlsanity.com/paper/2503.01505/) (2025)
- [Distributed Neural Representation for Reactive in situ Visualization](https://mlsanity.com/paper/2304.10516/) (2023)
- [SRN-SZ: Deep Leaning-Based Scientific Error-bounded Lossy Compression with Super-resolution Neural Networks](https://mlsanity.com/paper/2309.04037/) (2023)
- [Accelerating Communication in Deep Learning Recommendation Model Training with Dual-Level Adaptive Lossy Compression](https://mlsanity.com/paper/2407.04272/) (2024)
- [Attention Based Machine Learning Methods for Data Reduction with Guaranteed Error Bounds](https://mlsanity.com/paper/2409.05357/) (2024)
- [MC-INR: Efficient Encoding of Multivariate Scientific Simulation Data using Meta-Learning and Clustered Implicit Neural Representations](https://mlsanity.com/paper/2507.02494/) (2025)
- [Hybrid Neural Representations for Spherical Data](https://mlsanity.com/paper/2402.05965/) (2024)
- [NeurLZ: An Online Neural Learning-Based Method to Enhance Scientific Lossy Compression](https://mlsanity.com/paper/2409.05785/) (2024)
- [Guaranteed Conditional Diffusion: 3D Block-based Models for Scientific Data Compression](https://mlsanity.com/paper/2502.12951/) (2025)
- [Transforming Weather Data from Pixel to Latent Space](https://mlsanity.com/paper/2503.06623/) (2025)
- [F-Hash: Feature-Based Hash Design for Time-Varying Volume Visualization via Multi-Resolution Tesseract Encoding](https://mlsanity.com/paper/2507.03836/) (2025)
- [Machine Learning Techniques for Data Reduction of CFD Applications](https://mlsanity.com/paper/2404.18063/) (2024)
- [Foundation Model for Lossy Compression of Spatiotemporal Scientific Data](https://mlsanity.com/paper/2412.17184/) (2024)
- [ANTIC: Adaptive Neural Temporal In-situ Compressor](https://mlsanity.com/paper/2604.09543/) (2026)
- [Residual Modeling for High-Fidelity Learned Compression of Scientific Data](https://mlsanity.com/paper/2606.05389/) (2026)
- [Lessons Learned on the Path to Guaranteeing the Error Bound in Lossy Quantizers](https://mlsanity.com/paper/2407.15037/) (2024)
- [Enhancing Lossy Compression Through Cross-Field Information for Scientific Applications](https://mlsanity.com/paper/2409.18295/) (2024)
- [GraphComp: Extreme Error-bounded Compression of Scientific Data via Temporal Graph Autoencoders](https://mlsanity.com/paper/2505.06316/) (2025)
- [LLMComp: A Language Modeling Paradigm for Error-Bounded Scientific Data Compression (Technical Report)](https://mlsanity.com/paper/2510.23632/) (2025)
- [In Situ Training of Implicit Neural Compressors for Scientific Simulations via Sketch-Based Regularization](https://mlsanity.com/paper/2511.02659/) (2025)
- [DeepCQ: General-Purpose Deep-Surrogate Framework for Lossy Compression Quality Prediction](https://mlsanity.com/paper/2512.21433/) (2025)
- [Embedding Compression via Spherical Coordinates](https://mlsanity.com/paper/2602.00079/) (2026)
- [Mitigating Artifacts in Pre-quantization Based Scientific Data Compressors with Quantization-aware Interpolation](https://mlsanity.com/paper/2602.20097/) (2026)
- [EVOLVE: Efficient Learned Volume Compression with Variable-Rate Encoding on a Cross-Domain Database](https://mlsanity.com/paper/2607.18187/) (2026)
- [From Scalars to Time Series: Rethinking Implicit Neural Representations for Time-Varying Volumetric Data](https://mlsanity.com/paper/2607.20970/) (2026)
- [Cadence: Error-Bounded Lossy Compression of Demand Time Series with a Time-Series Foundation Model](https://mlsanity.com/paper/2609.06008/) (2026)

## Abstract (arXiv)

Today's scientific simulations require a significant reduction of data volume because of extremely large amounts of data they produce and the limited I/O bandwidth and storage space. Error-bounded lossy compressor has been considered one of the most effective solutions to the above problem. In practice, however, the best-fit compression method often needs to be customized/optimized in particular because of diverse characteristics in different datasets and various user requirements on the compression quality and performance. In this paper, we develop a novel modular, composable compression framework (namely SZ3), which involves three significant contributions. (1) SZ3 features a modular abstraction for the prediction-based compression framework such that the new compression modules can be plugged in easily. (2) SZ3 supports multialgorithm predictors and can automatically select the best-fit predictor for each data block based on the designed error estimation criterion. (3) SZ3 allows users to easily compose different compression pipelines on demand, such that both compression quality and performance can be significantly improved for their specific datasets and requirements. (4) In addition, we evaluate several lossy compressors composed from SZ3 using the real-world datasets. Specifically, we leverage SZ3 to improve the compression quality and performance for different use-cases, including GAMESS quantum chemistry dataset and Advanced Photon Source (APS) instrument dataset. Experiments show that our customized compression pipelines lead to up to 20% improvement in compression ratios under the same data distortion compared with the state-of-the-art approaches.