# Benchmarking TPU, GPU, and CPU Platforms for Deep Learning

arXiv: https://arxiv.org/abs/1907.10701 · HTML: https://mlsanity.com/paper/1907.10701/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jul 2019.

## Built on by (most cited first, 30 of 52)

- [Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyond](https://mlsanity.com/paper/2102.01998/) (2021)
- [A Practical Survey on Faster and Lighter Transformers](https://mlsanity.com/paper/2103.14636/) (2021)
- [Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI](https://mlsanity.com/paper/2111.06061/) (2021)
- [Fusion of Federated Learning and Industrial Internet of Things: A Survey](https://mlsanity.com/paper/2101.00798/) (2021)
- [A Machine Learning-oriented Survey on Tiny Machine Learning](https://mlsanity.com/paper/2309.11932/) (2023)
- [A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?](https://mlsanity.com/paper/2304.04780/) (2023)
- [Quantization and Deployment of Deep Neural Networks on Microcontrollers](https://mlsanity.com/paper/2105.13331/) (2021)
- [In Defense of Cross-Encoders for Zero-Shot Retrieval](https://mlsanity.com/paper/2212.06121/) (2022)
- [Prompt-prompted Adaptive Structured Pruning for Efficient LLM Generation](https://mlsanity.com/paper/2404.01365/) (2024)
- [A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes](https://mlsanity.com/paper/2102.06356/) (2021)
- [AI-native Interconnect Framework for Integration of Large Language Model Technologies in 6G Systems](https://mlsanity.com/paper/2311.05842/) (2023)
- [MLPerf HPC: A Holistic Benchmark Suite for Scientific Machine Learning on HPC Systems](https://mlsanity.com/paper/2110.11466/) (2021)
- [The Case for Co-Designing Model Architectures with Hardware](https://mlsanity.com/paper/2401.14489/) (2024)
- [Scalable Training of Language Models using JAX pjit and TPUv4](https://mlsanity.com/paper/2204.06514/) (2022)
- [Hybrid Neural Networks for On-device Directional Hearing](https://mlsanity.com/paper/2112.05893/) (2021)
- [EL-Attention: Memory Efficient Lossless Attention for Generation](https://mlsanity.com/paper/2105.04779/) (2021)
- [Which Student is Best? A Comprehensive Knowledge Distillation Exam for Task-Specific BERT Models](https://mlsanity.com/paper/2201.00558/) (2022)
- [Accelerating Neural Network Inference with Processing-in-DRAM: From the Edge to the Cloud](https://mlsanity.com/paper/2209.08938/) (2022)
- [MP3net: coherent, minute-long music generation from raw audio with a simple convolutional GAN](https://mlsanity.com/paper/2101.04785/) (2021)
- [Scientometric Review of Artificial Intelligence for Operations & Maintenance of Wind Turbines: The Past, Present and Future](https://mlsanity.com/paper/2204.02360/) (2022)
- [Systems for Parallel and Distributed Large-Model Deep Learning Training](https://mlsanity.com/paper/2301.02691/) (2023)
- [Accurate Block Quantization in LLMs with Outliers](https://mlsanity.com/paper/2403.20137/) (2024)
- [Simple and Efficient ways to Improve REALM](https://mlsanity.com/paper/2104.08710/) (2021)
- [Deep Learning on Edge TPUs](https://mlsanity.com/paper/2108.13732/) (2021)
- [Exploring the Impact of Virtualization on the Usability of the Deep Learning Applications](https://mlsanity.com/paper/2112.09780/) (2021)
- [Dataset and Benchmarking of Real-Time Embedded Object Detection for RoboCup SSL](https://mlsanity.com/paper/2106.14597/) (2021)
- [Accelerating Deep Learning Classification with Error-controlled Approximate-key Caching](https://mlsanity.com/paper/2112.06671/) (2021)
- [Block Format Error Bounds and Optimal Block Size Selection](https://mlsanity.com/paper/2210.05470/) (2022)
- [AccelGen: Heterogeneous SLO-Guaranteed High-Throughput LLM Inference Serving for Diverse Applications](https://mlsanity.com/paper/2503.13737/) (2025)
- [Deep Learning Models on CPUs: A Methodology for Efficient Training](https://mlsanity.com/paper/2206.10034/) (2022)

## Abstract (arXiv)

Training deep learning models is compute-intensive and there is an industry-wide trend towards hardware specialization to improve performance. To systematically benchmark deep learning platforms, we introduce ParaDnn, a parameterized benchmark suite for deep learning that generates end-to-end models for fully connected (FC), convolutional (CNN), and recurrent (RNN) neural networks. Along with six real-world models, we benchmark Google's Cloud TPU v2/v3, NVIDIA's V100 GPU, and an Intel Skylake CPU platform. We take a deep dive into TPU architecture, reveal its bottlenecks, and highlight valuable lessons learned for future specialized system design. We also provide a thorough comparison of the platforms and find that each has unique strengths for some types of models. Finally, we quantify the rapid performance improvements that specialized software stacks provide for the TPU and GPU platforms.