mlsanityData to 7 Oct 2026

FromPitch2Board: Benchmarking LLM Agents in Long-Horizon Football Management

arXiv 2609.34710 ↗First posted 28 Sep 2026cs.AIcited by 0

On the map: LLM agents and tool use → Financial trading and forecasting agents also fits: Social simulation and multi-agent debate

Abstract (arXiv)

Long-horizon agent benchmarks typically report how far an agent progresses, but do not identify whether its performance comes from the foundation model, scaffold, responsibility scope, match-control granularity, or horizon. We introduce FromPitch2Board, a deterministic football-management benchmark that studies five configurable factors through controlled comparisons on a single simulator, using paired seeds and a frozen calibration. We evaluate four foundation models and four agent scaffolds. In the Model Track, Coach points Z-scores span 0.19, while Manager points Z-scores span 0.68, with GPT-5.6 showing a sharp rise in passivity under responsibility expansion. Its responsibility ladder rises from 46.1 to 58.1 points with recruitment, then falls to 46.8 under full management, localizing the regression to the final responsibility boundary. Across that boundary, its skipped-decision rate rises from 1.1% to 57.9%. Within the Flash-Pro pair crossed across every scaffold, scaffold choice changes Manager points Z-scores by up to 0.48 relative to the fixed stateless scaffold. The 3Y cohort shows a directional reversal in mean ranking between years one and three, while a selected Claude Code+Pro configuration peaks in year three and remains below that peak, showing that responsibility scope and horizon expose behavior changes that a single headline score conceals.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Financial trading and forecasting agents, one of 16 directions in LLM agents and tool use. Builds on 38 of its 42 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

38 of 42 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026Cattle Trade: A Multi-Agent Benchmark for LLM Bluffing, Bidding, and Bargaining14 May
  2. 2025StockBench: Can LLM Agents Trade Stocks Profitably In Real-world Markets?2 Oct

Other references · 36

  1. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  2. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  3. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  4. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  5. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  6. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  7. 2023GAIA: a benchmark for General AI Assistantscited by 1,084
  8. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 656
  9. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
  10. 2018TextWorld: A Learning Environment for Text-based Gamescited by 304
  11. 2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 272
  12. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 257
24 more
  1. 2020The NetHack Learning Environmentcited by 196
  2. 2021Benchmarking the Spectrum of Agent Capabilitiescited by 193
  3. 2025MultiAgentBench: Evaluating the Collaboration and Competition of LLM agentscited by 169
  4. 2024BALROG: Benchmarking Agentic LLM and VLM Reasoning On Gamescited by 141
  5. 2023SmartPlay: A Benchmark for LLMs as Intelligent Agentscited by 123
  6. 2024AgentScope: A Flexible yet Robust Multi-Agent Platformcited by 120
  7. 2024GameBench: Evaluating Strategic Reasoning Abilities of LLM Agentscited by 89
  8. 2025Vending-Bench: A Benchmark for Long-Term Coherence of Autonomous Agentscited by 75
  9. 2026Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflowscited by 44
  10. 2026DeepPlanning: Benchmarking Long-Horizon Agentic Planning with Verifiable Constraintscited by 43
  11. 2025Orak: A Foundational Benchmark for Training and Evaluating LLM Agents on Diverse Video Gamescited by 30
  12. 2026Continual Harness: Online Adaptation for Self-Improving Foundation Agentscited by 30
  13. 2025UltraHorizon: Benchmarking Agent Capabilities in Ultra Long-Horizon Scenarioscited by 22
  14. 2026Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Gradingcited by 20
  15. 2021The AI Economist: Optimal Economic Policy Design via Two-level Deep Reinforcement Learningcited by 14
  16. 2026textttYC-Bench: Benchmarking AI Agents for Long-Term Planning and Consistent Executioncited by 10
  17. 2026The PokeAgent Challenge: Competitive and Long-Context Learning at Scalecited by 8
  18. 2026The Scaffold Effect in Coding Agents: Harness Choice as a Hidden Variable in Coding-Agent Evaluationcited by 8
  19. 2026CoffeeBench: Benchmarking Long-Horizon LLM Agents in Heterogeneous Multi-Agent Economiescited by 7
  20. 2026Agent Lightning v1.0: Towards Harnessed Agentic RLcited by 7
  21. 2026Agent Planning Benchmark: A Diagnostic Framework for Planning Capabilities in LLM Agentscited by 6
  22. 2026MerchantBench: Benchmarking LLM Agents for Long-Term Coherence in E-Commerce Operationscited by 5
  23. 2026CEO-Bench: Can Agents Play the Long Game?cited by 4
  24. 2026FM-Bench: A Benchmark for Long-Horizon Management with Competing Agentscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Static Leaderboards: Predictive Validity for the Evaluation of LLM Agents
  2. 2026From Score Matrices to Football-Aware Match-State Simulation: An Auditable LLM Harness for Exact-Score Reranking
  3. 2026TacticGen: Grounding Adaptable and Scalable Generation of Football Tactics
  4. 2026Leadership as Coordination Control: Behavioral Signatures and the Recovery-Advantage Boundary in Multi-Agent LLM Teams
  5. 2026The Double Measurement Confound in Agent Benchmarks: De-Scaffolding, Ground-Truth Scoring, and Reliability Beyond the Mean
  6. 2026Information Specialization and Constrained Synthesis in Multi-Agent LLM Forecasting: A Prospective Live-Study of the 2026 FIFA World Cup
  7. 2026When Agents Disagree: The Selection Bottleneck in Multi-Agent LLM Pipelines
  8. 2026State-Grounded Conditioning: Wrapping User-Facing LLM Agents Where Direction Depends on Live State