# Demystifying the Nvidia Ampere Architecture through Microbenchmarking and Instruction-level Analysis

arXiv: https://arxiv.org/abs/2208.11174 · HTML: https://mlsanity.com/paper/2208.11174/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Aug 2022.

## Built on by (most cited first, 9 of 9)

- [Scaling Laws for Precision](https://mlsanity.com/paper/2411.04330/) (2024)
- [MicroScopiQ: Accelerating Foundational Models through Outlier-Aware Microscaling Quantization](https://mlsanity.com/paper/2411.05282/) (2024)
- [QuEST: Stable Training of LLMs with 1-Bit Weights and Activations](https://mlsanity.com/paper/2502.05003/) (2025)
- [MixLLM: LLM Quantization with Global Mixed-precision between Output-features and Highly-efficient System Design](https://mlsanity.com/paper/2412.14590/) (2024)
- [HybridGen: Efficient LLM Generative Inference via CPU-GPU Hybrid Computing](https://mlsanity.com/paper/2604.18529/) (2026)
- [APT-LLM: Exploiting Arbitrary-Precision Tensor Core Computing for LLM Acceleration](https://mlsanity.com/paper/2508.19087/) (2025)
- [Fast NF4 Dequantization Kernels for Large Language Model Inference](https://mlsanity.com/paper/2604.02556/) (2026)
- [Incast-Free MoE Rate-Based Scheduling](https://mlsanity.com/paper/2607.26340/) (2026)
- [Scalable High-Fidelity Macromolecular Docking for GPU-Accelerated Supercomputers](https://mlsanity.com/paper/2608.07078/) (2026)

## Abstract (arXiv)

Graphics processing units (GPUs) are now considered the leading hardware to accelerate general-purpose workloads such as AI, data analytics, and HPC. Over the last decade, researchers have focused on demystifying and evaluating the microarchitecture features of various GPU architectures beyond what vendors reveal. This line of work is necessary to understand the hardware better and build more efficient workloads and applications. Many works have studied the recent Nvidia architectures, such as Volta and Turing, comparing them to their successor, Ampere. However, some microarchitecture features, such as the clock cycles for the different instructions, have not been extensively studied for the Ampere architecture. In this paper, we study the clock cycles per instructions with various data types found in the instruction-set architecture (ISA) of Nvidia GPUs. Using microbenchmarks, we measure the clock cycles for PTX ISA instructions and their SASS ISA instructions counterpart. we further calculate the clock cycle needed to access each memory unit. We also demystify the new version of the tensor core unit found in the Ampere architecture by using the WMMA API and measuring its clock cycles per instruction and throughput for the different data types and input shapes. The results found in this work should guide software developers and hardware architects. Furthermore, the clock cycles per instructions are widely used by performance modeling simulators and tools to model and predict the performance of the hardware.