mlsanityData to 14 Oct 2026

DMC4ML: Data Movement Complexity for Machine Learning

arXiv 2312.14441 ↗First posted 22 Dec 2023eess.SY cs.LG cs.SYcited by 0

Not on the map. Why?

Abstract (arXiv)

The greatest demand for today's computing is machine learning. This paper analyzes three machine learning algorithms: transformers, spatial convolution, and FFT. The analysis is novel in three aspects. First, it measures the cost of memory access on an abstract memory hierarchy, instead of traditional time or space complexity. Second, the analysis is asymptotic and identifies the primary sources of the memory cost. Finally, the result is symbolic, which can be used to select algorithmic parameters such as the group size in grouped query attention for any dimension size and number of heads and the batch size for batched convolution for any image size and kernel size.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 33 references that are in the map. First posted on arXiv 22 Dec 2023.

Builds on

10 of 33 references.

References · 10

  1. 2017Attention Is All You Needcited by 60,361
  2. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  3. 2019Searching for MobileNetV3cited by 1,994
  4. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  5. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 943
  6. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 485
  7. 2020Data Movement Is All You Need: A Case Study on Optimizing Transformerscited by 108
  8. 2015Fast Algorithms for Convolutional Neural Networkscited by 102
  9. 2019Communication Lower Bound in Convolution Acceleratorscited by 2
  10. 2022Beyond Time Complexity: Data Movement Complexity Analysis for Matrix Multiplicationcited by 2

Built on by

Not cited yet.