mlsanityData to 7 Oct 2026

Flexible Coded Distributed Convolution Computing for Enhanced Straggler Resilience and Numerical Stability in Distributed CNNs

arXiv 2411.01579 ↗First posted 3 Nov 2024cs.DC cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Deploying Convolutional Neural Networks (CNNs) on resource-constrained devices necessitates efficient management of computational resources, often via distributed environments susceptible to latency from straggler nodes. This paper introduces the Flexible Coded Distributed Convolution Computing (FCDCC) framework to enhance straggler resilience and numerical stability in distributed CNNs. We extend Coded Distributed Computing (CDC) with Circulant and Rotation Matrix Embedding (CRME) which was originally proposed for matrix multiplication to high-dimensional tensor convolution. For the proposed scheme, referred to as the Numerically Stable Coded Tensor Convolution (NSCTC) scheme, we also propose two new coded partitioning schemes: Adaptive-Padding Coded Partitioning (APCP) for the input tensor and Kernel-Channel Coded Partitioning (KCCP) for the filter tensor. These strategies enable linear decomposition of tensor convolutions and encoding them into CDC subtasks, combining model parallelism with coded redundancy for robust and efficient execution. Theoretical analysis identifies an optimal trade-off between communication and storage costs. Empirical results validate the framework's effectiveness in computational efficiency, straggler resilience, and scalability across various CNN architectures.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 17 of its 51 references that are in the map. First posted on arXiv 3 Nov 2024.

Builds on

17 of 51 references.

References · 17

  1. 2014How transferable are features in deep neural networks?cited by 1,216
  2. 2019Transfusion: Understanding Transfer Learning for Medical Imagingcited by 366
  3. 2014Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluationcited by 304
  4. 2015Fast Algorithms for Convolutional Neural Networkscited by 102
  5. 2015Speeding Up Distributed Machine Learning Using Codescited by 76
  6. 2018Lagrange Coded Computing: Optimal Design for Resiliency, Security and Privacycited by 66
  7. 2017CirCNN: Accelerating and Compressing Deep Neural Networks Using Block-CirculantWeight Matricescited by 53
  8. 2017Polynomial Codes: an Optimal Design for High-Dimensional Coded Matrix Multiplicationcited by 32
  9. 2021High Quality Segmentation for Ultra High-resolution Imagescited by 31
  10. 2014Fast Convolutional Nets With fbfft: A GPU Performance Evaluationcited by 27
  11. 2018On the Optimal Recovery Threshold of Coded Matrix Multiplicationcited by 27
  12. 2018Exploring Hidden Dimensions in Parallelizing Convolutional Neural Networkscited by 23
5 more
  1. 2018A Unified Coded Deep Neural Network Training Strategy Based on Generalized PolyDot Codes for Matrix Multiplicationcited by 8
  2. 2020Coded sparse matrix computation schemes that leverage partial stragglerscited by 5
  3. 2021Flexible Distributed Matrix Multiplicationcited by 2
  4. 2022Dynamic Coded Distributed Convolution for UAV-based Networked Airborne Computingcited by 2
  5. 2024Sparsity-Preserving Encodings for Straggler-Optimal Distributed Matrix Computations at the Edgecited by 1

Built on by

Not cited yet.