mlsanityData to 14 Oct 2026

Benchmarking TPU, GPU, and CPU Platforms for Deep Learning

arXiv 1907.10701 ↗First posted 24 Jul 2019cs.LG cs.PF stat.MLcited by 52

Not on the map. Why?

Abstract (arXiv)

Training deep learning models is compute-intensive and there is an industry-wide trend towards hardware specialization to improve performance. To systematically benchmark deep learning platforms, we introduce ParaDnn, a parameterized benchmark suite for deep learning that generates end-to-end models for fully connected (FC), convolutional (CNN), and recurrent (RNN) neural networks. Along with six real-world models, we benchmark Google's Cloud TPU v2/v3, NVIDIA's V100 GPU, and an Intel Skylake CPU platform. We take a deep dive into TPU architecture, reveal its bottlenecks, and highlight valuable lessons learned for future specialized system design. We also provide a thorough comparison of the platforms and find that each has unique strengths for some types of models. Finally, we quantify the rapid performance improvements that specialized software stacks provide for the TPU and GPU platforms.

Key facts. Cited by 52 papers in the map (data through 14 Oct 2026). First posted on arXiv 24 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 15; 2022: 12; 2023: 9; 2024: 6; 2025: 7; 2026: 3.

52 papers.

Sort:

By area:

  1. 2021Unbox the Black-box for the Medical Explainable AI via Multi-modal and Multi-centre Data Fusion: A Mini-Review, Two Showcases and Beyondcited by 76
  2. 2021A Practical Survey on Faster and Lighter Transformerscited by 73
  3. 2021Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AIcited by 39
  4. 2021Fusion of Federated Learning and Industrial Internet of Things: A Surveycited by 29
  5. 2023A Machine Learning-oriented Survey on Tiny Machine Learningcited by 27
  6. 2023A Review on Explainable Artificial Intelligence for Healthcare: Why, How, and When?cited by 24
  7. 2021Quantization and Deployment of Deep Neural Networks on Microcontrollerscited by 21
  8. 2022In Defense of Cross-Encoders for Zero-Shot Retrievalcited by 21
  9. 2024Prompt-prompted Adaptive Structured Pruning for Efficient LLM Generationcited by 21
  10. 2021A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizescited by 20
  11. 2023AI-native Interconnect Framework for Integration of Large Language Model Technologies in 6G Systemscited by 19
  12. 2021MLPerf HPC: A Holistic Benchmark Suite for Scientific Machine Learning on HPC Systemscited by 15
38 more
  1. 2024The Case for Co-Designing Model Architectures with Hardwarecited by 11
  2. 2022Scalable Training of Language Models using JAX pjit and TPUv4cited by 9
  3. 2021Hybrid Neural Networks for On-device Directional Hearingcited by 8
  4. 2021EL-Attention: Memory Efficient Lossless Attention for Generationcited by 6
  5. 2022Which Student is Best? A Comprehensive Knowledge Distillation Exam for Task-Specific BERT Modelscited by 5
  6. 2022Accelerating Neural Network Inference with Processing-in-DRAM: From the Edge to the Cloudcited by 5
  7. 2021MP3net: coherent, minute-long music generation from raw audio with a simple convolutional GANcited by 4
  8. 2022Scientometric Review of Artificial Intelligence for Operations & Maintenance of Wind Turbines: The Past, Present and Futurecited by 4
  9. 2023Systems for Parallel and Distributed Large-Model Deep Learning Trainingcited by 4
  10. 2024Accurate Block Quantization in LLMs with Outlierscited by 4
  11. 2021Simple and Efficient ways to Improve REALMcited by 3
  12. 2021Deep Learning on Edge TPUscited by 3
  13. 2021Exploring the Impact of Virtualization on the Usability of the Deep Learning Applicationscited by 3
  14. 2021Dataset and Benchmarking of Real-Time Embedded Object Detection for RoboCup SSLcited by 2
  15. 2021Accelerating Deep Learning Classification with Error-controlled Approximate-key Cachingcited by 2
  16. 2022Block Format Error Bounds and Optimal Block Size Selectioncited by 2
  17. 2025AccelGen: Heterogeneous SLO-Guaranteed High-Throughput LLM Inference Serving for Diverse Applicationscited by 2
  18. 2022Deep Learning Models on CPUs: A Methodology for Efficient Trainingcited by 1
  19. 2022Benchmarking GPU and TPU Performance with Graph Neural Networkscited by 1
  20. 2023IMAC-Sim: A Circuit-level Simulator For In-Memory Analog Computing Architecturescited by 1
  21. 2023Subgraph Stationary Hardware-Software Inference Co-Designcited by 1
  22. 2023Accelerator-driven Data Arrangement to Minimize Transformers Run-time on Multi-core Architecturescited by 1
  23. 2024Opportunities and challenges in the application of large artificial intelligence models in radiologycited by 1
  24. 2024Don't Think It Twice: Exploit Shift Invariance for Efficient Online Streaming Inference of CNNscited by 1
  25. 2025SMDP-Based Dynamic Batching for Improving Responsiveness and Energy Efficiency of Batch Servicescited by 1
  26. 2026DSTAR: Accelerating Diffusion Transformers via Spatial and Temporal Redundancy Reductioncited by 1
  27. 2022ScaLA: Accelerating Adaptation of Pre-Trained Transformer-Based Language Models via Efficient Large-Batch Adversarial Noisecited by 0
  28. 2022Minimizing Entropy to Discover Good Solutions to Recurrent Mixed Integer Programscited by 0
  29. 2022Recent Developments in AI and USPTO Open Datacited by 0
  30. 2022Improving the Performance of DNN-based Software Services using Automated Layer Cachingcited by 0
  31. 2023NeuralMatrix: Compute the Entire Neural Networks with Linear Matrix Operations for Efficient Inferencecited by 0
  32. 2023Benchmarking GPUs on SVBRDF Extractor Modelcited by 0
  33. 2024Universal on-chip polarization handling with deep photonic networkscited by 0
  34. 2025Energy-Efficient Dynamic Training and Inference for GNN-Based Network Modelingcited by 0
  35. 2025Deep Learning Model Deployment in Multiple Cloud Providers: an Exploratory Study Using Low Computing Power Environmentscited by 0
  36. 2025Input Resolution Downsizing as a Compression Technique for Vision Deep Learning Systemscited by 0
  37. 2025Estudio de la eficiencia en la escalabilidad de GPUs para el entrenamiento de Inteligencia Artificialcited by 0
  38. 2025FLAMMABLE: A Multi-Model Federated Learning Framework with Multi-Model Engagement and Adaptive Batch Sizescited by 0

50 most cited of 52 All 52 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025TPU-Gen: LLM-Driven Custom Tensor Processing Unit Generator
  2. 2023Performance Characterization of using Quantization for DNN Inference on Edge Devices: Extended Version
  3. 2026Gradient Under Microscope: Benchmarking Resource Utilization of Memory-Efficient Gradient Computation Methods
  4. 2025Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures
  5. 2021Accelerating SLIDE Deep Learning on Modern CPUs: Vectorization, Quantizations, Memory Optimizations, and More
  6. 2026Ragged Paged Attention: A High-Performance and Flexible LLM Inference Kernel for TPU
  7. 2025SparOA: Sparse and Operator-aware Hybrid Scheduling for Edge DNN Inference
  8. 2026SLAI T-Rex: Full-Parameter Post-training of the DeepSeek-V4 Family on Ascend SuperPOD