# Balancing Efficiency and Flexibility for DNN Acceleration via Temporal GPU-Systolic Array Integration

arXiv: https://arxiv.org/abs/2002.08326 · HTML: https://mlsanity.com/paper/2002.08326/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Feb 2020.

## Built on by (most cited first, 8 of 8)

- [OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantization](https://mlsanity.com/paper/2304.07493/) (2023)
- [SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximation](https://mlsanity.com/paper/2202.07471/) (2022)
- [ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantization](https://mlsanity.com/paper/2208.14286/) (2022)
- [Dual-side Sparse Tensor Core](https://mlsanity.com/paper/2105.09564/) (2021)
- [M-ANT: Efficient Low-bit Group Quantization for LLMs via Mathematically Adaptive Numerical Type](https://mlsanity.com/paper/2502.18755/) (2025)
- [CuBridge: An LLM-Based Framework for Understanding and Reconstructing High-Performance Attention Kernels](https://mlsanity.com/paper/2605.05023/) (2026)
- [AdaptGear: Accelerating GNN Training via Adaptive Subgraph-Level Kernels on GPUs](https://mlsanity.com/paper/2305.17408/) (2023)
- [A High-Performance Accelerator for Super-Resolution Processing on Embedded GPU](https://mlsanity.com/paper/2303.08999/) (2023)

## Abstract (arXiv)

The research interest in specialized hardware accelerators for deep neural networks (DNN) spikes recently owing to their superior performance and efficiency. However, today's DNN accelerators primarily focus on accelerating specific "kernels" such as convolution and matrix multiplication, which are vital but only part of an end-to-end DNN-enabled application. Meaningful speedups over the entire application often require supporting computations that are, while massively parallel, ill-suited to DNN accelerators. Integrating a general-purpose processor such as a CPU or a GPU incurs significant data movement overhead and leads to resource under-utilization on the DNN accelerators. We propose Simultaneous Multi-mode Architecture (SMA), a novel architecture design and execution model that offers general-purpose programmability on DNN accelerators in order to accelerate end-to-end applications. The key to SMA is the temporal integration of the systolic execution model with the GPU-like SIMD execution model. The SMA exploits the common components shared between the systolic-array accelerator and the GPU, and provides lightweight reconfiguration capability to switch between the two modes in-situ. The SMA achieves up to 63% performance improvement while consuming 23% less energy than the baseline Volta architecture with TensorCore.