# Flexible Coded Distributed Convolution Computing for Enhanced Straggler Resilience and Numerical Stability in Distributed CNNs

arXiv: https://arxiv.org/abs/2411.01579 · HTML: https://mlsanity.com/paper/2411.01579/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 51 references that are in the map.
- First posted on arXiv 3 Nov 2024.

## Builds on (most cited first, 17 of 17)

- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Transfusion: Understanding Transfer Learning for Medical Imaging](https://mlsanity.com/paper/1902.07208/) (2019)
- [Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation](https://mlsanity.com/paper/1404.0736/) (2014)
- [Fast Algorithms for Convolutional Neural Networks](https://mlsanity.com/paper/1509.09308/) (2015)
- [Speeding Up Distributed Machine Learning Using Codes](https://mlsanity.com/paper/1512.02673/) (2015)
- [Lagrange Coded Computing: Optimal Design for Resiliency, Security and Privacy](https://mlsanity.com/paper/1806.00939/) (2018)
- [CirCNN: Accelerating and Compressing Deep Neural Networks Using Block-CirculantWeight Matrices](https://mlsanity.com/paper/1708.08917/) (2017)
- [Polynomial Codes: an Optimal Design for High-Dimensional Coded Matrix Multiplication](https://mlsanity.com/paper/1705.10464/) (2017)
- [High Quality Segmentation for Ultra High-resolution Images](https://mlsanity.com/paper/2111.14482/) (2021)
- [Fast Convolutional Nets With fbfft: A GPU Performance Evaluation](https://mlsanity.com/paper/1412.7580/) (2014)
- [On the Optimal Recovery Threshold of Coded Matrix Multiplication](https://mlsanity.com/paper/1801.10292/) (2018)
- [Exploring Hidden Dimensions in Parallelizing Convolutional Neural Networks](https://mlsanity.com/paper/1802.04924/) (2018)
- [A Unified Coded Deep Neural Network Training Strategy Based on Generalized PolyDot Codes for Matrix Multiplication](https://mlsanity.com/paper/1811.10751/) (2018)
- [Coded sparse matrix computation schemes that leverage partial stragglers](https://mlsanity.com/paper/2012.06065/) (2020)
- [Flexible Distributed Matrix Multiplication](https://mlsanity.com/paper/2107.10448/) (2021)
- [Dynamic Coded Distributed Convolution for UAV-based Networked Airborne Computing](https://mlsanity.com/paper/2201.01431/) (2022)
- [Sparsity-Preserving Encodings for Straggler-Optimal Distributed Matrix Computations at the Edge](https://mlsanity.com/paper/2408.05152/) (2024)

## Abstract (arXiv)

Deploying Convolutional Neural Networks (CNNs) on resource-constrained devices necessitates efficient management of computational resources, often via distributed environments susceptible to latency from straggler nodes. This paper introduces the Flexible Coded Distributed Convolution Computing (FCDCC) framework to enhance straggler resilience and numerical stability in distributed CNNs. We extend Coded Distributed Computing (CDC) with Circulant and Rotation Matrix Embedding (CRME) which was originally proposed for matrix multiplication to high-dimensional tensor convolution. For the proposed scheme, referred to as the Numerically Stable Coded Tensor Convolution (NSCTC) scheme, we also propose two new coded partitioning schemes: Adaptive-Padding Coded Partitioning (APCP) for the input tensor and Kernel-Channel Coded Partitioning (KCCP) for the filter tensor. These strategies enable linear decomposition of tensor convolutions and encoding them into CDC subtasks, combining model parallelism with coded redundancy for robust and efficient execution. Theoretical analysis identifies an optimal trade-off between communication and storage costs. Empirical results validate the framework's effectiveness in computational efficiency, straggler resilience, and scalability across various CNN architectures.