# Packing Sparse Convolutional Neural Networks for Efficient Systolic Array Implementations: Column Combining Under Joint Optimization

arXiv: https://arxiv.org/abs/1811.04770 · HTML: https://mlsanity.com/paper/1811.04770/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Nov 2018.

## Built on by (most cited first, 15 of 15)

- [FAST: DNN Training Under Variable Precision Block Floating Point with Stochastic Rounding](https://mlsanity.com/paper/2110.15456/) (2021)
- [S2TA: Exploiting Structured Sparsity for Energy-Efficient Mobile CNN Acceleration](https://mlsanity.com/paper/2107.07983/) (2021)
- [Two Sparsities Are Better Than One: Unlocking the Performance Benefits of Sparse-Sparse Networks](https://mlsanity.com/paper/2112.13896/) (2021)
- [HighLight: Efficient and Flexible DNN Acceleration with Hierarchical Structured Sparsity](https://mlsanity.com/paper/2305.12718/) (2023)
- [VEGETA: Vertically-Integrated Extensions for Sparse/Dense GEMM Tile Acceleration on CPUs](https://mlsanity.com/paper/2302.08687/) (2023)
- [Accelerating DNN Training with Structured Data Gradient Pruning](https://mlsanity.com/paper/2202.00774/) (2022)
- [AI/ML Algorithms and Applications in VLSI Design and Technology](https://mlsanity.com/paper/2202.10015/) (2022)
- [Scale-out Systolic Arrays](https://mlsanity.com/paper/2203.11540/) (2022)
- [VersaGNN: a Versatile accelerator for Graph neural networks](https://mlsanity.com/paper/2105.01280/) (2021)
- [SMASH: Sparse Matrix Atomic Scratchpad Hashing](https://mlsanity.com/paper/2105.14156/) (2021)
- [S2Engine: A Novel Systolic Architecture for Sparse Convolutional Neural Networks](https://mlsanity.com/paper/2106.07894/) (2021)
- [Tight Compression: Compressing CNN Through Fine-Grained Pruning and Weight Permutation for Efficient Implementation](https://mlsanity.com/paper/2104.01303/) (2021)
- [SPA-GCN: Efficient and Flexible GCN Accelerator with an Application for Graph Similarity Computation](https://mlsanity.com/paper/2111.05936/) (2021)
- [Efficiera Residual Networks: Hardware-Friendly Fully Binary Weight with 2-bit Activation Model Achieves Practical ImageNet Accuracy](https://mlsanity.com/paper/2410.11553/) (2024)
- [ESCA: Enabling Seamless Codec Avatar Execution through Algorithm and Hardware Co-Optimization for Virtual Reality](https://mlsanity.com/paper/2510.24787/) (2025)

## Abstract (arXiv)

This paper describes a novel approach of packing sparse convolutional neural networks for their efficient systolic array implementations. By combining subsets of columns in the original filter matrix associated with a convolutional layer, we increase the utilization efficiency of the systolic array substantially (e.g., ~4x) due to the increased density of nonzeros in the resulting packed filter matrix. In combining columns, for each row, all filter weights but one with the largest magnitude are pruned. We retrain the remaining weights to preserve high accuracy. We demonstrate that in mitigating data privacy concerns the retraining can be accomplished with only fractions of the original dataset (e.g., 10% for CIFAR-10). We study the effectiveness of this joint optimization for both high utilization and classification accuracy with ASIC and FPGA designs based on efficient bit-serial implementations of multiplier-accumulators. We present analysis and empirical evidence on the superior performance of our column combining approach against prior arts under metrics such as energy efficiency (3x) and inference latency (12x).