# Exploiting Parallelism Opportunities with Deep Learning Frameworks

arXiv: https://arxiv.org/abs/1908.04705 · HTML: https://mlsanity.com/paper/1908.04705/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Aug 2019.

## Built on by (most cited first, 10 of 10)

- [AutoFL: Enabling Heterogeneity-Aware Energy Efficient Federated Learning](https://mlsanity.com/paper/2107.08147/) (2021)
- [RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation](https://mlsanity.com/paper/2201.10095/) (2022)
- [MLPerf HPC: A Holistic Benchmark Suite for Scientific Machine Learning on HPC Systems](https://mlsanity.com/paper/2110.11466/) (2021)
- [SECDA: Efficient Hardware/Software Co-Design of FPGA-based DNN Accelerators for Edge Inference](https://mlsanity.com/paper/2110.00478/) (2021)
- [Optimizing Inference Performance of Transformers on CPUs](https://mlsanity.com/paper/2102.06621/) (2021)
- [Packrat: Automatic Reconfiguration for Latency Minimization in CPU-based DNN Serving](https://mlsanity.com/paper/2311.18174/) (2023)
- [Automatic Tuning of Tensorflow's CPU Backend using Gradient-Free Optimization Algorithms](https://mlsanity.com/paper/2109.06266/) (2021)
- [Improving Inference Performance of Machine Learning with the Divide-and-Conquer Principle](https://mlsanity.com/paper/2301.05099/) (2023)
- [Automatic Task Parallelization of Dataflow Graphs in ML/DL models](https://mlsanity.com/paper/2308.11192/) (2023)
- [GEVO-ML: Optimizing Machine Learning Code with Evolutionary Computation](https://mlsanity.com/paper/2310.10211/) (2023)

## Abstract (arXiv)

State-of-the-art machine learning frameworks support a wide variety of design features to enable a flexible machine learning programming interface and to ease the programmability burden on machine learning developers. Identifying and using a performance-optimal setting in feature-rich frameworks, however, involves a non-trivial amount of performance profiling efforts and often relies on domain-specific knowledge. This paper takes a deep dive into analyzing the performance impact of key design features in a machine learning framework and quantifies the role of parallelism. The observations and insights distill into a simple set of guidelines that one can use to achieve much higher training and inference speedup. Across a diverse set of real-world deep learning models, the evaluation results show that the proposed performance tuning guidelines outperform the Intel and TensorFlow recommended settings by 1.29x and 1.34x, respectively.