# Benchmarking Deep Learning Convolutions on Energy-constrained CPUs

arXiv: https://arxiv.org/abs/2509.26217 · HTML: https://mlsanity.com/paper/2509.26217/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 33 references that are in the map.
- First posted on arXiv 30 Sep 2025.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Efficient Processing of Deep Neural Networks: A Tutorial and Survey](https://mlsanity.com/paper/1703.09039/) (2017)
- [YOLOv11: An Overview of the Key Architectural Enhancements](https://mlsanity.com/paper/2410.17725/) (2024)
- [Fast Algorithms for Convolutional Neural Networks](https://mlsanity.com/paper/1509.09308/) (2015)
- [High Performance Zero-Memory Overhead Direct Convolutions](https://mlsanity.com/paper/1809.10170/) (2018)
- [Anatomy Of High-Performance Deep Learning Convolutions On SIMD Architectures](https://mlsanity.com/paper/1808.05567/) (2018)
- [Characterizing and Demystifying the Implicit Convolution Algorithm on Commercial Matrix-Multiplication Accelerators](https://mlsanity.com/paper/2110.03901/) (2021)
- [Efficient Winograd Convolution via Integer Arithmetic](https://mlsanity.com/paper/1901.01965/) (2019)
- [Revisiting DETR Pre-training for Object Detection](https://mlsanity.com/paper/2308.01300/) (2023)
- [DeLTA: GPU Performance Model for Deep Learning Applications with In-depth Memory System Traffic Analysis](https://mlsanity.com/paper/1904.01691/) (2019)
- [Dalek: An Unconventional and Energy-Aware Heterogeneous Cluster](https://mlsanity.com/paper/2508.10481/) (2025)
- [Towards Effective Depthwise Convolutions on ARMv8 Architecture](https://mlsanity.com/paper/2206.12124/) (2022)

## Abstract (arXiv)

This work evaluates State-of-the-Art convolution algorithms for CPU-based CNN inference. Although most prior studies focus on GPUs or NPUs, CPU implementations remain comparatively under-optimized. Our first contribution is to provide fair benchmarking for embedded CPU inference. We evaluate direct, GEMM-based, and Winograd convolutions across modern CPUs from ARM, Intel, AMD, and NVIDIA vendors, considering both latency and energy efficiency. To the best of our knowledge, this is the first study to present a fair, cross-vendor comparison of CPU energy consumption using a high-resolution socket-level measurement platform. To validate our methodology, we further compare socket-level power measurements with estimates derived from model-specific registers (MSRs), finding that MSRs underestimate the power consumption of convolution inference by 10--30%. Our results show that the ARMR Cortex-A78AE CPU combined with an implicit GEMM convolution implementation offers the best trade-off between latency and power consumption, achieving ResNet50v1.5 inference in 102 ms with an average power of 25.3 W, corresponding to 2.58 J.