# Efficient Compression of Sparse Accelerator Data Using Implicit Neural Representations and Importance Sampling

arXiv: https://arxiv.org/abs/2412.01754 · HTML: https://mlsanity.com/paper/2412.01754/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 37 references that are in the map.
- First posted on arXiv 2 Dec 2024.

## Builds on (most cited first, 15 of 15)

- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [COIN: COmpression with Implicit Neural representations](https://mlsanity.com/paper/2103.03123/) (2021)
- [WIRE: Wavelet Implicit Neural Representations](https://mlsanity.com/paper/2301.05187/) (2023)
- [Improving Inference for Neural Image Compression](https://mlsanity.com/paper/2006.04240/) (2020)
- [Lossy Compression for Lossless Prediction](https://mlsanity.com/paper/2106.10800/) (2021)
- [Conditional Coding for Flexible Learned Video Compression](https://mlsanity.com/paper/2104.07930/) (2021)
- [Compression with Bayesian Implicit Neural Representations](https://mlsanity.com/paper/2305.19185/) (2023)
- [Modality-Agnostic Variational Compression of Implicit Neural Representations](https://mlsanity.com/paper/2301.09479/) (2023)
- [Lossless compression with state space models using bits back coding](https://mlsanity.com/paper/2103.10150/) (2021)
- [Efficient Data Compression for 3D Sparse TPC via Bicephalous Convolutional Autoencoder](https://mlsanity.com/paper/2111.05423/) (2021)
- [Accelerating Multigrid-based Hierarchical Scientific Data Refactoring on GPUs](https://mlsanity.com/paper/2007.04457/) (2020)
- [Reconstruction Distortion of Learned Image Compression with Imperceptible Perturbations](https://mlsanity.com/paper/2306.01125/) (2023)
- [Fast 2D Bicephalous Convolutional Autoencoder for Compressing 3D Time Projection Chamber Data](https://mlsanity.com/paper/2310.15026/) (2023)

## Abstract (arXiv)

High-energy, large-scale particle colliders in nuclear and high-energy physics generate data at extraordinary rates, reaching up to 1 terabyte and several petabytes per second, respectively. The development of real-time, high-throughput data compression algorithms capable of reducing this data to manageable sizes for permanent storage is of paramount importance. A unique characteristic of the tracking detector data is the extreme sparsity of particle trajectories in space, with an occupancy rate ranging from approximately 10-6 to 10%. Furthermore, for downstream tasks, a continuous representation of this data is often more useful than a voxel-based, discrete representation due to the inherently continuous nature of the signals involved. To address these challenges, we propose a novel approach using implicit neural representations for data learning and compression. We also introduce an importance sampling technique to accelerate the network training process. Our method is competitive with traditional compression algorithms, such as MGARD, SZ, and ZFP, while offering significant speed-ups and maintaining negligible accuracy loss through our importance sampling strategy.