# Systolic Arrays and Structured Pruning Co-design for Efficient Transformers in Edge Systems

arXiv: https://arxiv.org/abs/2411.10285 · HTML: https://mlsanity.com/paper/2411.10285/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 34 references that are in the map.
- First posted on arXiv 15 Nov 2024.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [ESPnet: End-to-End Speech Processing Toolkit](https://mlsanity.com/paper/1804.00015/) (2018)
- [Learning Structured Sparsity in Deep Neural Networks](https://mlsanity.com/paper/1608.03665/) (2016)
- [Pruning and Quantization for Deep Neural Network Acceleration: A Survey](https://mlsanity.com/paper/2101.09671/) (2021)
- [Efficient Transformer-based Large Scale Language Representations using Hardware-friendly Block Structured Pruning](https://mlsanity.com/paper/2009.08065/) (2020)
- [A Survey of Techniques for Optimizing Transformer Inference](https://mlsanity.com/paper/2307.07982/) (2023)
- [Accelerator-Aware Pruning for Convolutional Neural Networks](https://mlsanity.com/paper/1804.09862/) (2018)
- [Systolic Tensor Array: An Efficient Structured-Sparse GEMM Accelerator for Mobile CNN Inference](https://mlsanity.com/paper/2005.08098/) (2020)
- [Benchmarking Memory-Centric Computing Systems: Analysis of Real Processing-in-Memory Hardware](https://mlsanity.com/paper/2110.01709/) (2021)
- [CODEBench: A Neural Architecture and Hardware Accelerator Co-Design Framework](https://mlsanity.com/paper/2212.03965/) (2022)
- [SMAUG: End-to-End Full-Stack Simulation Infrastructure for Deep Learning Workloads](https://mlsanity.com/paper/1912.04481/) (2019)

## Built on by (most cited first, 1 of 1)

- [Precision-Aware Variable Bit Processing Elements for Hardware-Efficient Systolic Array Designs](https://mlsanity.com/paper/2608.22378/) (2026)

## Abstract (arXiv)

Efficient deployment of resource-intensive transformers on edge devices necessitates cross-stack optimization. We thus study the interrelation between structured pruning and systolic acceleration, matching the size of pruned blocks with the systolic array dimensions. In this setting, computations of pruned weight blocks can be skipped, reducing run-time and energy consumption, but potentially impacting quality of service (QoS). To evaluate the trade-offs between systolic array size and sparsity opportunities, we present a novel co-design framework that integrates algorithmic optimization, system simulation, and hardware design. Targeting speech recognition and machine translation using transformers as case study, we analyze how configuration choices across the stack affect performance metrics. Results demonstrate that structured pruning on systems featuring systolic array acceleration can effectively increase performance, while maintaining high QoS levels. Up to 44% system-wide speedups due to structured pruning and quantization were measured, with only 1.4% word error rate degradation on the standard LibriSpeech dataset.