mlsanityData to 7 Oct 2026

cuDNN: Efficient Primitives for Deep Learning

arXiv 1410.0759 ↗First posted 3 Oct 2014cs.NE cs.LG cs.MScited by 240

Not on the map. Why?

Abstract (arXiv)

We present a library of efficient implementations of deep learning primitives. Deep learning workloads are computationally intensive, and optimizing their kernels is difficult and time-consuming. As parallel architectures evolve, kernels must be reoptimized, which makes maintaining codebases difficult over time. Similar issues have long been addressed in the HPC community by libraries such as the Basic Linear Algebra Subroutines (BLAS). However, there is no analogous library for deep learning. Without such a library, researchers implementing deep learning workloads on parallel processors must create and optimize their own implementations of the main computational kernels, and this work must be repeated as new parallel processors emerge. To address this problem, we have created a library similar in intent to BLAS, with optimized routines for deep learning workloads. Our implementation contains routines for GPUs, although similarly to the BLAS library, these routines could be implemented for other platforms. The library is easy to integrate into existing frameworks, and provides optimized performance and memory usage. For example, integrating cuDNN into Caffe, a popular framework for convolutional networks, improves performance by 36% on a standard model while also reducing memory consumption.

Key facts. Cited by 240 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Oct 2014.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 64; 2022: 47; 2023: 39; 2024: 32; 2025: 26; 2026: 32.

240 papers.

Sort:

By area:

  1. 2021RepVGG: Making VGG-style ConvNets Great Againcited by 520
  2. 2021Scaling Local Self-Attention for Parameter Efficient Visual Backbonescited by 220
  3. 2024Vidur: A Large-Scale Simulation Framework For LLM Inferencecited by 115
  4. 2021Proof-of-Learning: Definitions and Practicecited by 81
  5. 2021Generalized Radiograph Representation Learning via Cross-supervision between Images and Free-text Radiology Reportscited by 80
  6. 2023Full Stack Optimization of Transformer Inference: a Surveycited by 79
  7. 2021Stochastic Training is Not Necessary for Generalizationcited by 63
  8. 2021Randomness In Neural Network Training: Characterizing The Impact of Toolingcited by 57
  9. 2025Towards Robust Agentic CUDA Kernel Benchmarking, Verification, and Optimizationcited by 52
  10. 2021Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threadscited by 51
  11. 2022Bamboo: Making Preemptible Instances Resilient for Affordable Training of Large DNNscited by 49
  12. 2024Efficient Deformable ConvNets: Rethinking Dynamic and Sparse Operator for Vision Applicationscited by 49
38 more
  1. 2021Nondeterminism and Instability in Neural Network Optimizationcited by 47
  2. 2025Astra: A Multi-Agent System for GPU Kernel Performance Optimizationcited by 46
  3. 2021LiBRe: A Practical Bayesian Approach to Adversarial Detectioncited by 40
  4. 2022Structural Pruning via Latency-Saliency Knapsackcited by 40
  5. 2024Mirage: A Multi-Level Superoptimizer for Tensor Programscited by 39
  6. 2022TensorIR: An Abstraction for Automatic Tensorized Program Optimizationcited by 38
  7. 2024Lightweight Deep Learning for Resource-Constrained Environments: A Surveycited by 38
  8. 2021Continuum: Simple Management of Complex Continual Learning Scenarioscited by 33
  9. 2022Enable Deep Learning on Mobile Devices: Methods, Systems, and Applicationscited by 32
  10. 2022Tensor Program Optimization with Probabilistic Programscited by 30
  11. 2021A Runtime-Based Computational Performance Predictor for Deep Neural Network Trainingcited by 29
  12. 2022Hidet: Task-Mapping Programming Paradigm for Deep Learning Tensor Programscited by 28
  13. 2022BiViT: Extremely Compressed Binary Vision Transformercited by 28
  14. 2021Dual-side Sparse Tensor Corecited by 27
  15. 2022Coarsening the Granularity: Towards Structurally Sparse Lottery Ticketscited by 27
  16. 2022Flashlight: Enabling Innovation in Tools for Machine Learningcited by 25
  17. 2022Towards Training Reproducible Deep Learning Modelscited by 23
  18. 2023FusionAI: Decentralized Training and Deploying LLMs with Massive Consumer-Level GPUscited by 20
  19. 2021"Adversarial Examples" for Proof-of-Learningcited by 19
  20. 2021RED++ : Data-Free Pruning of Deep Neural Networks via Input Splitting and Output Mergingcited by 19
  21. 2026AVO: Agentic Variation Operators for Autonomous Evolutionary Searchcited by 19
  22. 2021Anytime Dense Prediction with Confidence Adaptivitycited by 18
  23. 2021Bolt: Bridging the Gap between Auto-tuners and Hardware-native Performancecited by 18
  24. 2021TSM: Temporal Shift Module for Efficient and Scalable Video Understanding on Edge Devicecited by 17
  25. 2021The CoRa Tensor Compiler: Compilation for Ragged Tensors with Minimal Paddingcited by 17
  26. 2021Post-Training Sparsity-Aware Quantizationcited by 16
  27. 2021HELP: Hardware-Adaptive Efficient Latency Prediction for NAS via Meta-Learningcited by 16
  28. 2021Clairvoyant Prefetching for Distributed Machine Learning I/Ocited by 14
  29. 2021SparseDNN: Fast Sparse Deep Learning Inference on CPUscited by 13
  30. 2021Enabling Design Methodologies and Future Trends for Edge AI: Specialization and Co-designcited by 13
  31. 2021A compact butterfly-style silicon photonic-electronic neural chip for hardware-efficient deep learningcited by 13
  32. 2022Teal: Learning-Accelerated Optimization of WAN Traffic Engineeringcited by 13
  33. 2023Efficient On-device Training via Gradient Filteringcited by 13
  34. 2021Dynamic Precision Analog Computing for Neural Networkscited by 12
  35. 2021Real-Time Glaucoma Detection from Digital Fundus Images using Self-ONNscited by 12
  36. 2022Temporal Label Smoothing for Early Event Predictioncited by 12
  37. 2022Soft Masking for Cost-Constrained Channel Pruningcited by 12
  38. 2023Relax: Composable Abstractions for End-to-End Dynamic Machine Learningcited by 12

50 most cited of 240 All 240 (JSON)