# Eliminating Multi-GPU Performance Taxes: A Systems Approach to Efficient Distributed LLMs

arXiv: https://arxiv.org/abs/2511.02168 · HTML: https://mlsanity.com/paper/2511.02168/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 9 references that are in the map.
- First posted on arXiv 4 Nov 2025.

## Builds on (most cited first, 2 of 2)

- [Online normalizer calculation for softmax](https://mlsanity.com/paper/1805.02867/) (2018)
- [Triton-distributed: Programming Overlapping Kernels on Distributed AI Systems with the Triton Compiler](https://mlsanity.com/paper/2504.19442/) (2025)

## Built on by (most cited first, 2 of 2)

- [TaxBreak: Unmasking the Hidden Costs of LLM Inference Through Overhead Decomposition](https://mlsanity.com/paper/2603.12465/) (2026)
- [Iris: First-Class Multi-GPU Programming Experience in Triton](https://mlsanity.com/paper/2511.12500/) (2025)

## Abstract (arXiv)

As large language models (LLMs) continue to scale, their workloads increasingly rely on distributed execution across multiple GPUs. However, the conventional bulk synchronous parallel~(BSP) model used in such settings introduces significant performance inefficiencies. To characterize these bottlenecks, we introduce the ''Three Taxes'' (Bulk Synchronous, Inter-Kernel Data Locality, and Kernel Launch Overhead) as an analytical framework. We propose moving beyond the rigid BSP model to address key inefficiencies in distributed GPU execution. By exploiting libraries like Iris for Triton, we gain access to in-kernel communication primitives that enable the design of novel fine-grained programming patterns, offering greater flexibility and performance than traditional BSP-based approaches. These patterns systematically eliminate the three taxes by creating direct, tile-level producer-consumer pipelines and replacing global barriers with fine-grained dataflow synchronization. Applying this methodology to critical kernels, from the foundational All-Gather + general matrix multiplication operation to the complex Flash Decode algorithm, we observe a 10-20% speedup in end-to-end latency over BSP-based approaches, establishing a more programmable and efficient paradigm for distributed LLM workloads.