# Performance Analysis of Deep Learning Workloads on Leading-edge Systems

arXiv: https://arxiv.org/abs/1905.08764 · HTML: https://mlsanity.com/paper/1905.08764/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2019.

## Built on by (most cited first, 4 of 4)

- [Where Is My Training Bottleneck? Hidden Trade-Offs in Deep Learning Preprocessing Pipelines](https://mlsanity.com/paper/2202.08679/) (2022)
- [COMET: A Comprehensive Cluster Design Methodology for Distributed Deep Learning Training](https://mlsanity.com/paper/2211.16648/) (2022)
- [Hierarchical ML Codebook Design for Extreme MIMO Beam Management](https://mlsanity.com/paper/2312.02178/) (2023)
- [Enhancing Stability for Large Language Models Training in Constrained Bandwidth Networks](https://mlsanity.com/paper/2407.01614/) (2024)

## Abstract (arXiv)

This work examines the performance of leading-edge systems designed for machine learning computing, including the NVIDIA DGX-2, Amazon Web Services (AWS) P3, IBM Power System Accelerated Compute Server AC922, and a consumer-grade Exxact TensorEX TS4 GPU server. Representative deep learning workloads from the fields of computer vision and natural language processing are the focus of the analysis. Performance analysis is performed along with a number of important dimensions. Performance of the communication interconnects and large and high-throughput deep learning models are considered. Different potential use models for the systems as standalone and in the cloud also are examined. The effect of various optimization of the deep learning models and system configurations is included in the analysis.