# Performance Analysis of DNN Inference/Training with Convolution and non-Convolution Operations

arXiv: https://arxiv.org/abs/2306.16767 · HTML: https://mlsanity.com/paper/2306.16767/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 34 references that are in the map.
- First posted on arXiv 29 Jun 2023.

## Builds on (most cited first, 13 of 13)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [Federated Learning for Mobile Keyboard Prediction](https://mlsanity.com/paper/1811.03604/) (2018)
- [Gemmini: Enabling Systematic Deep-Learning Architecture Evaluation via Full-Stack Integration](https://mlsanity.com/paper/1911.09925/) (2019)
- [An Evaluation of Edge TPU Accelerators for Convolutional Neural Networks](https://mlsanity.com/paper/2102.10423/) (2021)
- [Interstellar: Using Halide's Scheduling Language to Analyze DNN Accelerators](https://mlsanity.com/paper/1809.04070/) (2018)
- [AutoDNNchip: An Automated DNN Chip Predictor and Builder for Both FPGAs and ASICs](https://mlsanity.com/paper/2001.03535/) (2020)
- [Training on the Edge: The why and the how](https://mlsanity.com/paper/1903.03051/) (2019)
- [Characterizing and Demystifying the Implicit Convolution Algorithm on Commercial Matrix-Multiplication Accelerators](https://mlsanity.com/paper/2110.03901/) (2021)
- [Now that I can see, I can improve: Enabling data-driven finetuning of CNNs on the edge](https://mlsanity.com/paper/2006.08554/) (2020)
- [A Highly Configurable Hardware/Software Stack for DNN Inference Acceleration](https://mlsanity.com/paper/2111.15024/) (2021)
- [TRIM: A Design Space Exploration Model for Deep Neural Networks Inference and Training Accelerators](https://mlsanity.com/paper/2105.08239/) (2021)

## Abstract (arXiv)

Today's performance analysis frameworks for deep learning accelerators suffer from two significant limitations. First, although modern convolutional neural network (CNNs) consist of many types of layers other than convolution, especially during training, these frameworks largely focus on convolution layers only. Second, these frameworks are generally targeted towards inference, and lack support for training operations. This work proposes a novel performance analysis framework, SimDIT, for general ASIC-based systolic hardware accelerator platforms. The modeling effort of SimDIT comprehensively covers convolution and non-convolution operations of both CNN inference and training on a highly parameterizable hardware substrate. SimDIT is integrated with a backend silicon implementation flow and provides detailed end-to-end performance statistics (i.e., data access cost, cycle counts, energy, and power) for executing CNN inference and training workloads. SimDIT-enabled performance analysis reveals that on a 64X64 processing array, non-convolution operations constitute 59.5% of total runtime for ResNet-50 training workload. In addition, by optimally distributing available off-chip DRAM bandwidth and on-chip SRAM resources, SimDIT achieves 18X performance improvement over a generic static resource allocation for ResNet-50 inference.