mlsanityData to 14 Oct 2026

PyTorch: An Imperative Style, High-Performance Deep Learning Library

arXiv 1912.01703 ↗First posted 3 Dec 2019cs.LG cs.MS stat.MLcited by 20,305

Not on the map. Why?

Cited by 601 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Deep learning frameworks have often focused on either usability or speed, but not both. PyTorch is a machine learning library that shows that these two goals are in fact compatible: it provides an imperative and Pythonic programming style that supports code as a model, makes debugging easy and is consistent with other popular scientific computing libraries, while remaining efficient and supporting hardware accelerators such as GPUs. In this paper, we detail the principles that drove the implementation of PyTorch and how they are reflected in its architecture. We emphasize that every aspect of PyTorch is a regular Python program under the full control of its user. We also explain how the careful and pragmatic implementation of the key components of its runtime enables them to work together to achieve compelling performance. We demonstrate the efficiency of individual subsystems, as well as the overall speed of PyTorch on several common benchmarks.

Key facts. Cited by 20,305 papers in the map (data through 14 Oct 2026). First posted on arXiv 3 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 4; 2021: 3363; 2022: 3883; 2023: 4216; 2024: 3746; 2025: 2988; 2026: 2105.

20,305 papers.

Sort:

By area:

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2022Flow Matching for Generative Modelingcited by 5,702
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  7. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  8. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  9. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  10. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  11. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  12. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
38 more
  1. 2022A ConvNet for the 2020scited by 3,579
  2. 2023A Survey of Large Language Modelscited by 2,576
  3. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  4. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  5. 2021Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecastingcited by 1,676
  6. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  7. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  8. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  9. 2022TensoRF: Tensorial Radiance Fieldscited by 1,280
  10. 2022FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecastingcited by 1,210
  11. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  12. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  13. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
  14. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,157
  15. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  16. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  17. 2022Editing Models with Task Arithmeticcited by 1,120
  18. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,101
  19. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  20. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  21. 2022TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysiscited by 1,058
  22. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
  23. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  24. 2022Token Merging: Your ViT But Fastercited by 1,021
  25. 2022Mass-Editing Memory in a Transformercited by 1,001
  26. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  27. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 931
  28. 2023iTransformer: Inverted Transformers Are Effective for Time Series Forecastingcited by 931
  29. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 927
  30. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  31. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
  32. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 899
  33. 2023Improving and generalizing flow-based generative models with minibatch optimal transportcited by 879
  34. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
  35. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  36. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
  37. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 827
  38. 2021Robust fine-tuning of zero-shot modelscited by 797

50 most cited of 20,305 All 20,305 (JSON)