mlsanityData to 14 Oct 2026

TorchBench: Benchmarking PyTorch with High API Surface Coverage

arXiv 2304.14226 ↗First posted 27 Apr 2023cs.LG cs.AI cs.PFcited by 2

Not on the map. Why?

Abstract (arXiv)

Deep learning (DL) has been a revolutionary technique in various domains. To facilitate the model development and deployment, many deep learning frameworks are proposed, among which PyTorch is one of the most popular solutions. The performance of ecosystem around PyTorch is critically important, which saves the costs of training models and reduces the response time of model inferences. In this paper, we propose TorchBench, a novel benchmark suite to study the performance of PyTorch software stack. Unlike existing benchmark suites, TorchBench encloses many representative models, covering a large PyTorch API surface. TorchBench is able to comprehensively characterize the performance of the PyTorch software stack, guiding the performance optimization across models, PyTorch framework, and GPU libraries. We show two practical use cases of TorchBench. (1) We profile TorchBench to identify GPU performance inefficiencies in PyTorch. We are able to optimize many performance bugs and upstream patches to the official PyTorch repository. (2) We integrate TorchBench into PyTorch continuous integration system. We are able to identify performance regression in multiple daily code checkins to prevent PyTorch repository from introducing performance bugs. TorchBench is open source and keeps evolving.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 41 of its 110 references that are in the map. First posted on arXiv 27 Apr 2023.

Builds on

41 of 110 references.

References · 41

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  7. 2017Mask R-CNNcited by 7,277
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  9. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  10. 2022Generative Adversarial Networkscited by 5,627
  11. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  12. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
29 more
  1. 2018YOLOv3: An Incremental Improvementcited by 2,659
  2. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  3. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  4. 2019Searching for MobileNetV3cited by 1,994
  5. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  6. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  7. 2017Mixed Precision Trainingcited by 1,156
  8. 2016Pruning Filters for Efficient ConvNetscited by 1,143
  9. 2017StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translationcited by 1,039
  10. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
  11. 2019Green AIcited by 627
  12. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  13. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 521
  14. 2015Variational Dropout and the Local Reparameterization Trickcited by 490
  15. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
  16. 2014Caffe: Convolutional Architecture for Fast Feature Embeddingcited by 288
  17. 2014cuDNN: Efficient Primitives for Deep Learningcited by 245
  18. 2016Trained Ternary Quantizationcited by 208
  19. 2015MXNet: A Flexible and Efficient Machine Learning Library for Heterogeneous Distributed Systemscited by 195
  20. 2022Hybrid Transformers for Music Source Separationcited by 134
  21. 2019MLPerf Training Benchmarkcited by 115
  22. 2020Background Matting: The World is Your Green Screencited by 105
  23. 2019Learning to Paint With Model-based Deep Reinforcement Learningcited by 71
  24. 2021DNNFusion: Accelerating Deep Neural Networks Execution with Advanced Operator Fusioncited by 57
  25. 2020Torch-Struct: Deep Structured Prediction Librarycited by 40
  26. 2016Fathom: Reference Workloads for Modern Deep Learning Methodscited by 20
  27. 2020Deep Slow Motion Video Reconstruction with Hybrid Imaging Systemcited by 14
  28. 2017Training Deep AutoEncoders for Collaborative Filteringcited by 10
  29. 2019AIBench: An Industry Standard Internet Service AI Benchmark Suitecited by 6

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2024Understanding the Performance Horizon of the Latest ML Workloads with NonGEMM Workloadscited by 4
  2. 2025Overcoming Labelled Data Scarcity for Defect Classification in Scanning Tunneling Microscopycited by 0