# Hierarchical Roofline Performance Analysis for Deep Learning Applications

arXiv: https://arxiv.org/abs/2009.05257 · HTML: https://mlsanity.com/paper/2009.05257/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Sep 2020.

## Built on by (most cited first, 11 of 11)

- [Model-Preserving Adaptive Rounding](https://mlsanity.com/paper/2505.22988/) (2025)
- [TPU-KNN: K Nearest Neighbor Search at Peak FLOP/s](https://mlsanity.com/paper/2206.14286/) (2022)
- [The Price of Freedom: Exploring Expressivity and Runtime Tradeoffs in Equivariant Tensor Products](https://mlsanity.com/paper/2506.13523/) (2025)
- [FastGRPO: Accelerating Policy Optimization via Concurrency-aware Speculative Decoding and Online Draft Learning](https://mlsanity.com/paper/2509.21792/) (2025)
- [EL-Attention: Memory Efficient Lossless Attention for Generation](https://mlsanity.com/paper/2105.04779/) (2021)
- [COMET: A Comprehensive Cluster Design Methodology for Distributed Deep Learning Training](https://mlsanity.com/paper/2211.16648/) (2022)
- [Equinox: Holistic Fair Scheduling in Serving Large Language Models](https://mlsanity.com/paper/2508.16646/) (2025)
- [Hierarchical Muon: Tiled Newton-Schulz Updates for Efficient Muon Optimization](https://mlsanity.com/paper/2606.27216/) (2026)
- [The Sparsity Roofline: Understanding the Hardware Limits of Sparse Neural Networks](https://mlsanity.com/paper/2310.00496/) (2023)
- [ShuntServe: Cost-Efficient LLM Serving on Heterogeneous Spot GPU Clusters](https://mlsanity.com/paper/2606.18600/) (2026)
- [A Distributed Framework for Causal Modeling of Performance Variability in GPU Traces](https://mlsanity.com/paper/2510.18300/) (2025)

## Abstract (arXiv)

This paper presents a practical methodology for collecting performance data necessary to conduct hierarchical Roofline analysis on NVIDIA GPUs. It discusses the extension of the Empirical Roofline Toolkit for broader support of a range of data precisions and Tensor Core support and introduces a Nsight Compute based method to accurately collect application performance information. This methodology allows for automated machine characterization and application characterization for Roofline analysis across the entire memory hierarchy on NVIDIA GPUs, and it is validated by a complex deep learning application used for climate image segmentation. We use two versions of the code, in TensorFlow and PyTorch respectively, to demonstrate the use and effectiveness of this methodology. We highlight how the application utilizes the compute and memory capabilities on the GPU and how the implementation and performance differ in two deep learning frameworks.