mlsanityData to 7 Oct 2026

NVIDIA Tensor Core Programmability, Performance & Precision

arXiv 1803.04014 ↗First posted 11 Mar 2018cs.DC cs.PFcited by 77

Not on the map. Why?

Abstract (arXiv)

The NVIDIA Volta GPU microarchitecture introduces a specialized unit, called "Tensor Core" that performs one matrix-multiply-and-accumulate on 4x4 matrices per clock cycle. The NVIDIA Tesla V100 accelerator, featuring the Volta microarchitecture, provides 640 Tensor Cores with a theoretical peak performance of 125 Tflops/s in mixed precision. In this paper, we investigate current approaches to program NVIDIA Tensor Cores, their performances and the precision loss due to computation in mixed precision. Currently, NVIDIA provides three different ways of programming matrix-multiply-and-accumulate on Tensor Cores: the CUDA Warp Matrix Multiply Accumulate (WMMA) API, CUTLASS, a templated library based on WMMA, and cuBLAS GEMM. After experimenting with different approaches, we found that NVIDIA Tensor Cores can deliver up to 83 Tflops/s in mixed precision on a Tesla V100 GPU, seven and three times the performance in single and half precision respectively. A WMMA implementation of batched GEMM reaches a performance of 4 Tflops/s. While precision loss due to matrix multiplication with half precision input might be critical in many HPC applications, it can be considerably reduced at the cost of increased computation. Our results indicate that HPC applications using matrix multiplications can strongly benefit from using of NVIDIA Tensor Cores.

Key facts. Cited by 77 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 9; 2023: 9; 2024: 17; 2025: 16; 2026: 15.

77 papers.

Sort:

By area:

  1. 2024Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Modelscited by 232
  2. 2022LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Modelscited by 98
  3. 2021Proof-of-Learning: Definitions and Practicecited by 81
  4. 2024MixDQ: Memory-Efficient Few-Step Text-to-Image Diffusion Models with Metric-Decoupled Mixed Precision Quantizationcited by 42
  5. 2024Mirage: A Multi-Level Superoptimizer for Tensor Programscited by 39
  6. 2023XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languagescited by 34
  7. 2021Fast Neural Representations for Direct Volume Renderingcited by 32
  8. 2024Jetfire: Efficient and Accurate Transformer Pretraining with INT8 Data Flow and Per-Block Quantizationcited by 30
  9. 2023A Machine Learning-oriented Survey on Tiny Machine Learningcited by 26
  10. 2022Flashlight: Enabling Innovation in Tools for Machine Learningcited by 25
  11. 2024Quantization-Aware Imitation-Learning for Resource-Efficient Robotic Controlcited by 25
  12. 2021PQK: Model Compression via Pruning, Quantization, and Knowledge Distillationcited by 19
38 more
  1. 2022TPU-KNN: K Nearest Neighbor Search at Peak FLOP/scited by 18
  2. 2021Post-Training Sparsity-Aware Quantizationcited by 16
  3. 2024Allo: A Programming Model for Composable Accelerator Designcited by 16
  4. 2021Clairvoyant Prefetching for Distributed Machine Learning I/Ocited by 14
  5. 2024PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculationcited by 14
  6. 2024MARCA: Mamba Accelerator with ReConfigurable Architecturecited by 13
  7. 2021SECDA: Efficient Hardware/Software Co-Design of FPGA-based DNN Accelerators for Edge Inferencecited by 12
  8. 2022DHEN: A Deep and Hierarchical Ensemble Network for Large-Scale Click-Through Rate Predictioncited by 12
  9. 2022Three-dimensional microstructure generation using generative adversarial neural networks in the context of continuum micromechanicscited by 11
  10. 2021Reference-based Defect Detection Networkcited by 10
  11. 2023Design Space Exploration of Low-Bit Quantized Neural Networks for Visual Place Recognitioncited by 10
  12. 2024Universal Checkpointing: A Flexible and Efficient Distributed Checkpointing System for Large-Scale DNN Training with Reconfigurable Paralleliscited by 9
  13. 2025QuZO: Quantized Zeroth-Order Fine-Tuning for Large Language Modelscited by 9
  14. 2025Saliency-Aware Quantized Imitation Learning for Efficient Robotic Controlcited by 8
  15. 2021Horizontally Fused Training Array: An Effective Hardware Utilization Squeezer for Training Novel Deep Learning Modelscited by 7
  16. 2021Low-Precision Hardware Architectures Meet Recommendation Model Inference at Scalecited by 7
  17. 2024Ascend HiFloat8 Format for Deep Learningcited by 7
  18. 2023ArchGym: An Open-Source Gymnasium for Machine Learning Assisted Architecture Designcited by 6
  19. 2022Quant 4.0: Engineering Quantitative Investment with Automated, Explainable and Knowledge-driven Artificial Intelligencecited by 5
  20. 2025QiMeng-TensorOp: Automatically Generating High-Performance Tensor Operators with Hardware Primitivescited by 5
  21. 2026MoE-SpAc: Efficient MoE Inference Based on Speculative Activation Utility in Heterogeneous Edge Scenarioscited by 5
  22. 2022GROW: A Row-Stationary Sparse-Dense GEMM Accelerator for Memory-Efficient Graph Convolutional Neural Networkscited by 4
  23. 2024FlattenQuant: Breaking Through the Inference Compute-bound for Large Language Models with Per-tensor Quantizationcited by 4
  24. 2025ReaLM: Reliable and Efficient Large Language Model Inference with Statistical Algorithm-Based Fault Tolerancecited by 4
  25. 2025QiMeng-Attention: SOTA Attention Operator is generated by SOTA Attention Algorithmcited by 4
  26. 2026PRISM: Distribution-free Adaptive Computation of Matrix Functions for Accelerating Neural Network Trainingcited by 4
  27. 2023Convolutional Networks with Oriented 1D Kernelscited by 3
  28. 2024FastAttention: Extend FlashAttention2 to NPUs and Low-resource GPUscited by 3
  29. 2021Optimization of Heterogeneous Systems with AI Planning Heuristics and Machine Learning: A Performance and Energy Aware Approachcited by 2
  30. 2024AI in Space for Scientific Missions: Strategies for Minimizing Neural-Network Model Uploadcited by 2
  31. 2024Debunking the CUDA Myth Towards GPU-based AI Systemscited by 2
  32. 2025BLaST: High Performance Inference and Pretraining using BLock Sparse Transformerscited by 2
  33. 2026BubbleSpec: Turning Long-Tail Bubbles into Speculative Rollout Drafts for Synchronous Reinforcement Learningcited by 2
  34. 2021Representation range needs for 16-bit neural network trainingcited by 1
  35. 2022EcoFlow: Efficient Convolutional Dataflows for Low-Power Neural Network Acceleratorscited by 1
  36. 2023RedBit: An End-to-End Flexible Framework for Evaluating the Accuracy of Quantized CNNscited by 1
  37. 2023LoopTune: Optimizing Tensor Computations with Reinforcement Learningcited by 1
  38. 2024depyf: Open the Opaque Box of PyTorch Compiler for Machine Learning Researcherscited by 1

50 most cited of 77 All 77 (JSON)