# PatDNN: Achieving Real-Time DNN Execution on Mobile Devices with Pattern-based Weight Pruning

arXiv: https://arxiv.org/abs/2001.00138 · HTML: https://mlsanity.com/paper/2001.00138/

## Key facts

- Cited by 76 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jan 2020.

## Built on by (most cited first, 30 of 76)

- [SPViT: Enabling Faster Vision Transformers via Soft Token Pruning](https://mlsanity.com/paper/2112.13890/) (2021)
- [MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edge](https://mlsanity.com/paper/2110.14032/) (2021)
- [EdgeMoE: Empowering Sparse Large Language Models on Mobile Devices](https://mlsanity.com/paper/2308.14352/) (2023)
- [Fast On-device LLM Inference with NPUs](https://mlsanity.com/paper/2407.05858/) (2024)
- [DNNFusion: Accelerating Deep Neural Networks Execution with Advanced Operator Fusion](https://mlsanity.com/paper/2108.13342/) (2021)
- [Advancing Model Pruning via Bi-level Optimization](https://mlsanity.com/paper/2210.04092/) (2022)
- [SparCL: Sparse Continual Learning on the Edge](https://mlsanity.com/paper/2209.09476/) (2022)
- [Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models](https://mlsanity.com/paper/2503.06027/) (2025)
- [LLMCad: Fast and Scalable On-device Large Language Model Inference](https://mlsanity.com/paper/2309.04255/) (2023)
- [Teachers Do More Than Teach: Compressing Image-to-Image Models](https://mlsanity.com/paper/2103.03467/) (2021)
- [Achieving on-Mobile Real-Time Super-Resolution with Neural Architecture and Pruning Search](https://mlsanity.com/paper/2108.08910/) (2021)
- [Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications](https://mlsanity.com/paper/2204.11786/) (2022)
- [Coarsening the Granularity: Towards Structurally Sparse Lottery Tickets](https://mlsanity.com/paper/2202.04736/) (2022)
- [Effective Model Sparsification by Scheduled Grow-and-Prune Methods](https://mlsanity.com/paper/2106.09857/) (2021)
- [EdgeFM: Leveraging Foundation Model for Open-set Learning on the Edge](https://mlsanity.com/paper/2311.10986/) (2023)
- [Compiler-Aware Neural Architecture Search for On-Mobile Real-time Super-Resolution](https://mlsanity.com/paper/2207.12577/) (2022)
- [Real-time Neural Network Inference on Extremely Weak Devices: Agile Offloading with Explainable AI](https://mlsanity.com/paper/2312.14229/) (2023)
- [Towards Artificial General Intelligence (AGI) in the Internet of Things (IoT): Opportunities and Challenges](https://mlsanity.com/paper/2309.07438/) (2023)
- [Enabling Design Methodologies and Future Trends for Edge AI: Specialization and Co-design](https://mlsanity.com/paper/2103.15750/) (2021)
- [HighLight: Efficient and Flexible DNN Acceleration with Hierarchical Structured Sparsity](https://mlsanity.com/paper/2305.12718/) (2023)
- [SmartFRZ: An Efficient Training Framework using Attention-Based Layer Freezing](https://mlsanity.com/paper/2401.16720/) (2024)
- [Layer Freezing & Data Sieving: Missing Pieces of a Generic Framework for Sparse Training](https://mlsanity.com/paper/2209.11204/) (2022)
- [GRIM: A General, Real-Time Deep Learning Inference Framework for Mobile Devices based on Fine-Grained Structured Weight Sparsity](https://mlsanity.com/paper/2108.11033/) (2021)
- [Hardware-Aware DNN Compression via Diverse Pruning and Mixed-Precision Quantization](https://mlsanity.com/paper/2312.15322/) (2023)
- [Dancing along Battery: Enabling Transformer with Run-time Reconfigurability on Mobile Devices](https://mlsanity.com/paper/2102.06336/) (2021)
- [ULEEN: A Novel Architecture for Ultra Low-Energy Edge Neural Networks](https://mlsanity.com/paper/2304.10618/) (2023)
- [Enabling Resource-efficient AIoT System with Cross-level Optimization: A survey](https://mlsanity.com/paper/2309.15467/) (2023)
- [Automatic Mapping of the Best-Suited DNN Pruning Schemes for Real-Time Mobile Acceleration](https://mlsanity.com/paper/2111.11581/) (2021)
- [LilNetX: Lightweight Networks with EXtreme Model Compression and Structured Sparsification](https://mlsanity.com/paper/2204.02965/) (2022)
- [Slim-neck by GSConv: A lightweight-design for real-time detector architectures](https://mlsanity.com/paper/2206.02424/) (2022)

## Abstract (arXiv)

With the emergence of a spectrum of high-end mobile devices, many applications that formerly required desktop-level computation capability are being transferred to these devices. However, executing the inference of Deep Neural Networks (DNNs) is still challenging considering high computation and storage demands, specifically, if real-time performance with high accuracy is needed. Weight pruning of DNNs is proposed, but existing schemes represent two extremes in the design space: non-structured pruning is fine-grained, accurate, but not hardware friendly; structured pruning is coarse-grained, hardware-efficient, but with higher accuracy loss. In this paper, we introduce a new dimension, fine-grained pruning patterns inside the coarse-grained structures, revealing a previously unknown point in design space. With the higher accuracy enabled by fine-grained pruning patterns, the unique insight is to use the compiler to re-gain and guarantee high hardware efficiency. In other words, our method achieves the best of both worlds, and is desirable across theory/algorithm, compiler, and hardware levels. The proposed PatDNN is an end-to-end framework to efficiently execute DNN on mobile devices with the help of a novel model compression technique (pattern-based pruning based on extended ADMM solution framework) and a set of thorough architecture-aware compiler- and code generation-based optimizations (filter kernel reordering, compressed weight storage, register load redundancy elimination, and parameter auto-tuning). Evaluation results demonstrate that PatDNN outperforms three state-of-the-art end-to-end DNN frameworks, TensorFlow Lite, TVM, and Alibaba Mobile Neural Network with speedup up to 44.5x, 11.4x, and 7.1x, respectively, with no accuracy compromise. Real-time inference of representative large-scale DNNs (e.g., VGG-16, ResNet-50) can be achieved using mobile devices.