# The Price of Anarchy in Disaggregated Inference

arXiv: https://arxiv.org/abs/2606.17081 · HTML: https://mlsanity.com/paper/2606.17081/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 40 references that are in the map.
- First posted on arXiv 11 Jun 2026.

## Builds on (most cited first, 17 of 17)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [SGLang: Efficient Execution of Structured Language Model Programs](https://mlsanity.com/paper/2312.07104/) (2023)
- [DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving](https://mlsanity.com/paper/2401.09670/) (2024)
- [FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU](https://mlsanity.com/paper/2303.06865/) (2023)
- [Splitwise: Efficient generative LLM inference using phase splitting](https://mlsanity.com/paper/2311.18677/) (2023)
- [Vidur: A Large-Scale Simulation Framework For LLM Inference](https://mlsanity.com/paper/2405.05465/) (2024)
- [Themis: Fair and Efficient GPU Cluster Scheduling](https://mlsanity.com/paper/1907.01484/) (2019)
- [Prefill-Decode Aggregation or Disaggregation? Unifying Both for Goodput-Optimized LLM Serving](https://mlsanity.com/paper/2508.01989/) (2025)
- [Shockwave: Fair and Efficient Cluster Scheduling for Dynamic Adaptation in Machine Learning](https://mlsanity.com/paper/2210.00093/) (2022)
- [Serving Long-Context LLMs at the Mobile Edge: Test-Time Reinforcement Learning-based Model Caching and Inference Offloading](https://mlsanity.com/paper/2501.14205/) (2025)
- [AIConfigurator: Lightning-Fast Configuration Optimization for Multi-Framework LLM Serving](https://mlsanity.com/paper/2601.06288/) (2026)
- [KV Pareto: Systems-Level Optimization of KV Cache and Model Compression for Long Context Inference](https://mlsanity.com/paper/2512.01953/) (2025)
- [Incentives in Dominant Resource Fair Allocation under Dynamic Demands](https://mlsanity.com/paper/2109.12401/) (2021)
- [Approximation and Convergence of Large Atomic Congestion Games](https://mlsanity.com/paper/2001.02797/) (2020)
- [Selfish Caching Games on Directed Graphs](https://mlsanity.com/paper/2012.14148/) (2020)
- [A Game-Theoretic Framework for Distributed Load Balancing: Static and Dynamic Game Models](https://mlsanity.com/paper/2501.15324/) (2025)

## Built on by (most cited first, 1 of 1)

- [Unified AI Gateway: A Framework for Joint Model Routing and KV Cache Management](https://mlsanity.com/paper/2609.06940/) (2026)

## Abstract (arXiv)

Disaggregated inference architectures physically separate prefill and decode phases onto distinct GPU pools, creating competing "agents" that share a fixed hardware budget. We provide, to our knowledge, the first formal game-theoretic analysis of this architecture, using NVIDIA Dynamo as a concrete case study. We model disaggregated serving as three coupled games: a two-player resource game between prefill and decode pools, a selfish caching game over the hierarchical KV cache, and a congestion game with positive externalities for request routing. We empirically validate the latter two; the P/D resource game is treated analytically (Section 9.2). We characterize how GPU saturation induces regime transitions that shift the game's payoff structure: below saturation, selfish behavior has bounded Price of Anarchy (PoA); at saturation, superlinear latency and cache externalities drive our empirical estimator PoA-hat (defined in Section 6.4) upward. Based on this analysis, we design an adaptive controller that detects saturation transitions in real time and adjusts routing parameters accordingly, shifting from cache-affinity exploitation to load-balanced congestion avoidance. We instantiate our framework on a 3-node NVIDIA B200 cluster running Dynamo with two models, Nemotron-4-340B (TP=8, full-node workers with cross-InfiniBand KV transfers) and Llama-3.1-70B (TP=4), and find the same three-regime PoA-hat structure with the same first post-knee grid point (C=128) on both models. Adaptive routing shifts each model to a better operating point. Our strongest result is on the 70B 1P/5D topology, where PoA-hat drops 3.1x (66.4 to 21.5) in the saturated phase at a 13% throughput cost. On the 70B 1P/2D, PoA-hat drops 2.2x and TTFT P99 drops 7.6x (see Section 8.5).