# SparsePixels: Efficient Convolution for Sparse Data on FPGAs

arXiv: https://arxiv.org/abs/2512.06208 · HTML: https://mlsanity.com/paper/2512.06208/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 63 references that are in the map.
- First posted on arXiv 5 Dec 2025.

## Builds on (most cited first, 27 of 27)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [FINN: A Framework for Fast, Scalable Binarized Neural Network Inference](https://mlsanity.com/paper/1612.07119/) (2016)
- [Fast inference of deep neural networks in FPGAs for particle physics](https://mlsanity.com/paper/1804.06913/) (2018)
- [Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors](https://mlsanity.com/paper/2006.10159/) (2020)
- [TorchSparse: Efficient Point Cloud Inference Engine](https://mlsanity.com/paper/2204.10319/) (2022)
- [Sparse 3D convolutional neural networks](https://mlsanity.com/paper/1505.02890/) (2015)
- [A survey of sparse representation: algorithms and applications](https://mlsanity.com/paper/1602.07017/) (2016)
- [Fast convolutional neural networks on FPGAs with hls4ml](https://mlsanity.com/paper/2101.05108/) (2021)
- [Jet-Images -- Deep Learning Edition](https://mlsanity.com/paper/1511.05190/) (2015)
- [NullHop: A Flexible Convolutional Neural Network Accelerator Based on Sparse Representations of Feature Maps](https://mlsanity.com/paper/1706.01406/) (2017)
- [TorchSparse++: Efficient Training and Inference Framework for Sparse Convolution on GPUs](https://mlsanity.com/paper/2311.12862/) (2023)
- [The Dark Machines Anomaly Score Challenge: Benchmark Data and Model Independent Event Classification for the Large Hadron Collider](https://mlsanity.com/paper/2105.14027/) (2021)
- [A Convolutional Neural Network Neutrino Event Classifier](https://mlsanity.com/paper/1604.01444/) (2016)
- [Fast inference of Boosted Decision Trees in FPGAs for particle physics](https://mlsanity.com/paper/2002.02534/) (2020)
- [The National Research Platform: Stretched, Multi-Tenant, Scientific Kubernetes Cluster](https://mlsanity.com/paper/2505.22864/) (2025)
- [Scalable Deep Convolutional Neural Networks for Sparse, Locally Dense Liquid Argon Time Projection Chamber Data](https://mlsanity.com/paper/1903.05663/) (2019)
- [An Efficient Hardware Accelerator for Structured Sparse Convolutional Neural Networks on FPGAs](https://mlsanity.com/paper/2001.01955/) (2020)
- [Ultra-low latency recurrent neural network inference on FPGAs for physics applications with hls4ml](https://mlsanity.com/paper/2207.00559/) (2022)
- [Real-time semantic segmentation on FPGAs for autonomous vehicles with hls4ml](https://mlsanity.com/paper/2205.07690/) (2022)
- [Symbolic Regression on FPGAs for Fast Machine Learning Inference](https://mlsanity.com/paper/2305.04099/) (2023)
- [SymbolNet: Neural Symbolic Regression with Adaptive Dynamic Pruning for Compression](https://mlsanity.com/paper/2401.09949/) (2024)
- [Real-time Anomaly Detection at the L1 Trigger of CMS Experiment](https://mlsanity.com/paper/2411.19506/) (2024)
- [Fast Neural Network Inference on FPGAs for Triggering on Long-Lived Particles at Colliders](https://mlsanity.com/paper/2307.05152/) (2023)
- [Low Latency Transformer Inference on FPGAs for Physics Applications with hls4ml](https://mlsanity.com/paper/2409.05207/) (2024)

## Abstract (arXiv)

Inference of standard convolutional neural networks (CNNs) on FPGAs often incurs high latency and a long initiation interval due to the deep nested loops required to densely convolve every input pixel regardless of its feature value. However, input features can be spatially sparse in some image data, where semantic information may occupy only a small fraction of the pixels and most computation would be wasted on empty regions. In this work, we introduce SparsePixels, a framework that implements sparse convolution on FPGAs by selectively retaining and computing on a small subset of active input pixels while ignoring the rest, reducing the per-layer compute from O(HWK2CinCout) to O(n2CinCout) and the storage from O(HWC) to O(nC) when n2ll HW. Because computation always runs over a single pre-specified pixel budget that is frozen at synthesis, the inference latency is data-independent and constant at runtime. We show that, for identifying neutrino interactions in naturally sparse LArTPC images with 4k pixels, a standard CNN with a compact size of 4k parameters incurs an inference latency of 48.665 μs on an FPGA, whereas a sparse CNN of the same base architecture, computing on less than 1% of the input pixels, achieves a × 73 speedup to 0.665 μs with resource utilization well within on-chip budgets, trading only a small percent-level performance loss. This work aims to benefit future algorithm development for efficient data readout in modern experiments with strict latency requirements of microseconds or below.