# Efficient distributed algorithms for Convolutional Neural Networks

arXiv: https://arxiv.org/abs/2105.13480 · HTML: https://mlsanity.com/paper/2105.13480/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 May 2021.

## Built on by (most cited first, 1 of 1)

- [TabConv: Low-Computation CNN Inference via Table Lookups](https://mlsanity.com/paper/2404.05872/) (2024)

## Abstract (arXiv)

Several efficient distributed algorithms have been developed for matrix-matrix multiplication: the 3D algorithm, the 2D SUMMA algorithm, and the 2.5D algorithm. Each of these algorithms was independently conceived and they trade-off memory needed per node and the inter-node data communication volume. The convolutional neural network (CNN) computation may be viewed as a generalization of matrix-multiplication combined with neighborhood stencil computations. We develop communication-efficient distributed-memory algorithms for CNNs that are analogous to the 2D/2.5D/3D algorithms for matrix-matrix multiplication.