Same Pieces, Different Servers: A Tetris Benchmark for AI Agents as Served

arXiv 2603.02348 ↗First posted 2 Mar 2026cs.LG cs.AI cs.ROcited by 0

Not on the map. Why?

Abstract (arXiv)

An agent meets a model as served: through an endpoint with a price card, a shared cache and other tenants, or on whatever hardware a self-hosted model runs. Benchmarks rank the weights. We introduce a Tetris benchmark that measures what agents get from models as served: every move is scored against an oracle, and every agent receives the same pieces. In five pre-specified experiments with nine open-weight models on one serverless provider, plus an open decision model self-hosted on a CPU, we find that price and size do not predict decision quality; that resending history costs almost nothing when cached input is free, would cost eleven to twelve times more if it were not, and makes play worse; that deployments keep between 4 and more than 64 agent contexts warm, in line with their throughput rather than the model's KV-cache size; that an agent's own long requests slow its slowest short decisions more than tenfold, which a simple admission rule cuts by 59% without hurting play; and that the decision model plays mid-pack but takes 27 s per move on a CPU, about 650 times longer than reported on a GPU. Architecture predicts some of what an agent sees; the deployment sets the rest.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 37 references that are in the map. First posted on arXiv 2 Mar 2026.

Builds on

22 of 37 references.

References · 22

2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
2025Qwen3 Technical Reportcited by 8,820
2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
2023AgentBench: Evaluating LLMs as Agentscited by 1,267
2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 815
2024DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Servingcited by 617
2023Splitwise: Efficient generative LLM inference using phase splittingcited by 558
2024Gated Delta Networks: Improving Mamba2 with Delta Rulecited by 532

10 more

2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 526
2019MLPerf Inference Benchmarkcited by 199
2024Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttentioncited by 163
2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Gamescited by 146
2025LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inferencecited by 126
2025Autellix: An Efficient Serving Engine for LLM Agents as General Programscited by 58
2025lmgame-Bench: How Good are LLMs at Playing Games?cited by 55
2025Are You Getting What You Pay For? Auditing Model Substitution in LLM APIscited by 38
2025Auditing Prompt Caching in Language Model APIscited by 20
2019The Game of Tetris in Machine Learningcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way