mlsanityData to 14 Oct 2026

The Price of Anarchy in Disaggregated Inference

arXiv 2606.17081 ↗First posted 11 Jun 2026cs.AR cs.AI cs.DCcited by 1

Not on the map. Why?

Abstract (arXiv)

Disaggregated inference architectures physically separate prefill and decode phases onto distinct GPU pools, creating competing "agents" that share a fixed hardware budget. We provide, to our knowledge, the first formal game-theoretic analysis of this architecture, using NVIDIA Dynamo as a concrete case study. We model disaggregated serving as three coupled games: a two-player resource game between prefill and decode pools, a selfish caching game over the hierarchical KV cache, and a congestion game with positive externalities for request routing. We empirically validate the latter two; the P/D resource game is treated analytically (Section 9.2). We characterize how GPU saturation induces regime transitions that shift the game's payoff structure: below saturation, selfish behavior has bounded Price of Anarchy (PoA); at saturation, superlinear latency and cache externalities drive our empirical estimator PoA-hat (defined in Section 6.4) upward. Based on this analysis, we design an adaptive controller that detects saturation transitions in real time and adjusts routing parameters accordingly, shifting from cache-affinity exploitation to load-balanced congestion avoidance. We instantiate our framework on a 3-node NVIDIA B200 cluster running Dynamo with two models, Nemotron-4-340B (TP=8, full-node workers with cross-InfiniBand KV transfers) and Llama-3.1-70B (TP=4), and find the same three-regime PoA-hat structure with the same first post-knee grid point (C=128) on both models. Adaptive routing shifts each model to a better operating point. Our strongest result is on the 70B 1P/5D topology, where PoA-hat drops 3.1x (66.4 to 21.5) in the saturated phase at a 13% throughput cost. On the 70B 1P/2D, PoA-hat drops 2.2x and TTFT P99 drops 7.6x (see Section 8.5).

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 17 of its 40 references that are in the map. First posted on arXiv 11 Jun 2026.

Builds on

17 of 40 references.

References · 17

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  4. 2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 617
  5. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 575
  6. 2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
  7. 2024Vidur: A Large-Scale Simulation Framework For LLM Inferencecited by 117
  8. 2019Themis: Fair and Efficient GPU Cluster Schedulingcited by 43
  9. 2025Prefill-Decode Aggregation or Disaggregation? Unifying Both for Goodput-Optimized LLM Servingcited by 23
  10. 2022Shockwave: Fair and Efficient Cluster Scheduling for Dynamic Adaptation in Machine Learningcited by 12
  11. 2025Serving Long-Context LLMs at the Mobile Edge: Test-Time Reinforcement Learning-based Model Caching and Inference Offloadingcited by 10
  12. 2026AIConfigurator: Lightning-Fast Configuration Optimization for Multi-Framework LLM Servingcited by 9
5 more
  1. 2025KV Pareto: Systems-Level Optimization of KV Cache and Model Compression for Long Context Inferencecited by 4
  2. 2021Incentives in Dominant Resource Fair Allocation under Dynamic Demandscited by 2
  3. 2020Approximation and Convergence of Large Atomic Congestion Gamescited by 1
  4. 2020Selfish Caching Games on Directed Graphscited by 1
  5. 2025A Game-Theoretic Framework for Distributed Load Balancing: Static and Dynamic Game Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Unified AI Gateway: A Framework for Joint Model Routing and KV Cache Managementcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026When Does Disaggregation Pay? Simulating Prefill--Decode--Attention--FFN Specialization for Agentic LLM Inference
  2. 2026PEARL: Adaptive Prefill-Decode Execution with Elasticity for Agentic Reinforcement Learning
  3. 2026CONCUR: High-Throughput Agentic Batch Inference of LLM via Congestion-Based Concurrency Control
  4. 2025Speculative Actions: A Lossless Framework for Faster Agentic Systems
  5. 2026DualPath: Breaking the Storage Bandwidth Bottleneck in Agentic LLM Inference
  6. 2026Memory Scarcity, Open Models, and the Restructuring of the AI Industry, 2026-2030 -- A quantitative scenario analysis of inference economics, training-cost divergence, and infrastructure solvency
  7. 2025RollArt: Disaggregated Multi-Task Agentic RL Training at Scale
  8. 2026TaxBreak: Unmasking the Hidden Costs of LLM Inference Through Overhead Decomposition