mlsanityData to 13 Oct 2026

Instant GPU Efficiency Visibility at Fleet Scale

arXiv 2605.20799 ↗First posted 20 May 2026cs.DC cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

We present Overall FLOP Utilization (OFU), a hardware-level, precision-agnostic GPU efficiency metric for AI workloads on HPC systems, derived from two on-chip performance counters: Tensor Pipe Activity and SM clock frequency. OFU requires no application instrumentation and works across GPU generations and numeric precisions. We characterize five properties of the OFU approximation -- tile quantization, floating-point precision scaling, clock sampling noise, Tensor Core clock domains, and non-tensor undercounting -- through controlled GEMM experiments on H100 and GB200 across FP16, TF32, FP8, and NVFP4. After tile-quantization correction, OFU predicts application-level MFU to within <=2 percentage points. Against 608 production training jobs, OFU achieves r = 0.78 correlation with application-level MFU and surfaces two framework-level FLOPs miscalculations. Deployed across large-scale GPU fleets, OFU has detected a 2.5x efficiency regression and tracked precision-dependent utilization changes in mixed-precision pretraining. Our evaluation and operational experience suggest OFU is a practical, deployment-ready complement to application-level MFU for continuous fleet-wide efficiency monitoring.

Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Builds on 10 of its 34 references that are in the map. First posted on arXiv 20 May 2026.

Builds on

10 of 34 references.

References · 10

  1. 2020Scaling Laws for Neural Language Modelscited by 6,494
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,317
  3. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,658
  4. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,115
  5. 2016Training Deep Nets with Sublinear Memory Costcited by 840
  6. 2022Reducing Activation Recomputation in Large Transformer Modelscited by 301
  7. 2019MLPerf Training Benchmarkcited by 114
  8. 2024MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUscited by 111
  9. 2020Data Movement Is All You Need: A Case Study on Optimizing Transformerscited by 108
  10. 2025Pretraining Large Language Models with NVFP4cited by 71

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Not to Break, but to Attest: Adversarial Probes for Privacy-Preserving LLM Verificationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Automated Tensor Scheduling for Hybrid CPU-GPU LLM Inference on Consumer Devices
  2. 2026Are LLM-Generated GPU Kernels Production-Ready? A Trace-Driven Benchmark and Optimization Agent
  3. 2026Decomposing Runtime, Kernel, and Quantization Speedups via a Matched FP16 Intermediate: A Hardware-Conditioned Case Study on Four NVIDIA RTX A5000 GPUs
  4. 2026Energy Efficiency of Locally Deployed LLMs: A Preliminary Quantitative GPU Power Benchmark on Consumer Hardware
  5. 2026The 1/W Law: An Analytical Study of Context-Length Routing Topology and GPU Generation Gains for LLM Inference Energy Efficiency
  6. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision
  7. 2025Static Batching of Irregular Workloads on GPUs: Framework and Application to Efficient MoE Model Inference
  8. 2025FAST: An Efficient Scheduler for All-to-All GPU Communication