# PERMDNN: Efficient Compressed DNN Architecture with Permuted Diagonal Matrices

arXiv: https://arxiv.org/abs/2004.10936 · HTML: https://mlsanity.com/paper/2004.10936/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Apr 2020.

## Built on by (most cited first, 12 of 12)

- [CHIP: CHannel Independence-based Pruning for Compact Neural Networks](https://mlsanity.com/paper/2110.13981/) (2021)
- [A Survey on Deep Learning Hardware Accelerators for Heterogeneous HPC Platforms](https://mlsanity.com/paper/2306.15552/) (2023)
- [Tensor Decomposition for Model Reduction in Neural Networks: A Review](https://mlsanity.com/paper/2304.13539/) (2023)
- [Doping: A technique for efficient compression of LSTM models using sparse structured additive matrices](https://mlsanity.com/paper/2102.07071/) (2021)
- [unzipFPGA: Enhancing FPGA-based CNN Engines with On-the-Fly Weights Generation](https://mlsanity.com/paper/2103.05600/) (2021)
- [How to Reach Real-Time AI on Consumer Devices? Solutions for Programmable and Custom Architectures](https://mlsanity.com/paper/2106.15021/) (2021)
- [Mitigating Memory Wall Effects in CNN Engines with On-the-Fly Weights Generation](https://mlsanity.com/paper/2307.13412/) (2023)
- [Tight Compression: Compressing CNN Through Fine-Grained Pruning and Weight Permutation for Efficient Implementation](https://mlsanity.com/paper/2104.01303/) (2021)
- [Mixture-of-Rookies: Saving DNN Computations by Predicting ReLU Outputs](https://mlsanity.com/paper/2202.04990/) (2022)
- [DNA-TEQ: An Adaptive Exponential Quantization of Tensors for DNN Inference](https://mlsanity.com/paper/2306.16430/) (2023)
- [A Vision Toward Energy-Efficient Domain-Specific Artificial Intelligence Models and Agents](https://mlsanity.com/paper/2510.22052/) (2025)
- [Two-Valued Symmetric Circulant Matrices: Applications in Deep Learning](https://mlsanity.com/paper/2605.16443/) (2026)

## Abstract (arXiv)

Deep neural network (DNN) has emerged as the most important and popular artificial intelligent (AI) technique. The growth of model size poses a key energy efficiency challenge for the underlying computing platform. Thus, model compression becomes a crucial problem. However, the current approaches are limited by various drawbacks. Specifically, network sparsification approach suffers from irregularity, heuristic nature and large indexing overhead. On the other hand, the recent structured matrix-based approach (i.e., CirCNN) is limited by the relatively complex arithmetic computation (i.e., FFT), less flexible compression ratio, and its inability to fully utilize input sparsity. To address these drawbacks, this paper proposes PermDNN, a novel approach to generate and execute hardware-friendly structured sparse DNN models using permuted diagonal matrices. Compared with unstructured sparsification approach, PermDNN eliminates the drawbacks of indexing overhead, non-heuristic compression effects and time-consuming retraining. Compared with circulant structure-imposing approach, PermDNN enjoys the benefits of higher reduction in computational complexity, flexible compression ratio, simple arithmetic computation and full utilization of input sparsity. We propose PermDNN architecture, a multi-processing element (PE) fully-connected (FC) layer-targeted computing engine. The entire architecture is highly scalable and flexible, and hence it can support the needs of different applications with different model configurations. We implement a 32-PE design using CMOS 28nm technology. Compared with EIE, PermDNN achieves 3.3x~4.8x higher throughout, 5.9x~8.5x better area efficiency and 2.8x~4.0x better energy efficiency on different workloads. Compared with CirCNN, PermDNN achieves 11.51x higher throughput and 3.89x better energy efficiency.