mlsanityData to 13 Oct 2026

LLM Swiss Round: Aggregating Multi-Benchmark Performance via Competitive Swiss-System Dynamics

arXiv 2512.21010 ↗First posted 24 Dec 2025cs.LG cs.AI cs.PFcited by 0

Not on the map. Why?

Abstract (arXiv)

The rapid proliferation of Large Language Models (LLMs) and diverse specialized benchmarks necessitates a shift from fragmented, task-specific metrics to a holistic, competitive ranking system that effectively aggregates performance across multiple ability dimensions. Primarily using static scoring, current evaluation methods are fundamentally limited. They struggle to determine the proper mix ratio across diverse benchmarks, and critically, they fail to capture a model's dynamic competitive fitness or its vulnerability when confronted with sequential, high-stakes tasks. To address this, we introduce the novel Competitive Swiss-System Dynamics (CSD) framework. CSD simulates a multi-round, sequential contest where models are dynamically paired across a curated sequence of benchmarks based on their accumulated win-loss record. And Monte Carlo Simulation (N=100,000 iterations) is used to approximate the statistically robust Expected Win Score (E[Sm]), which eliminates the noise of random pairing and early-round luck. Furthermore, we implement a Failure Sensitivity Analysis by parameterizing the per-round elimination quantity (Tk), which allows us to profile models based on their risk appetite--distinguishing between robust generalists and aggressive specialists. We demonstrate that CSD provides a more nuanced and context-aware ranking than traditional aggregate scoring and static pairwise models, representing a vital step towards risk-informed, next-generation LLM evaluation.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Builds on 29 of its 33 references that are in the map. First posted on arXiv 24 Dec 2025.

Builds on

29 of 33 references.

References · 29

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,876
  2. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,274
  3. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,804
  4. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,951
  5. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,799
  6. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,268
  7. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,191
  8. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,163
  9. 2023GAIA: a benchmark for General AI Assistantscited by 1,102
  10. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 628
  11. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 582
  12. 2025Humanity's Last Examcited by 478
17 more
  1. 2024Measuring short-form factuality in large language modelscited by 358
  2. 2025ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systemscited by 181
  3. 2025MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMscited by 175
  4. 2025Multi-SWE-bench: A Multilingual Benchmark for Issue Resolvingcited by 166
  5. 2024Long-form factuality in large language modelscited by 149
  6. 2025BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinesecited by 140
  7. 2025CodeElo: Benchmarking Competition-level Code Generation of LLMs with Human-comparable Elo Ratingscited by 111
  8. 2024KOR-Bench: Benchmarking Language Models on Knowledge-Orthogonal Reasoning Taskscited by 68
  9. 2025PHYBench: Holistic Evaluation of Physical Perception and Reasoning in Large Language Modelscited by 57
  10. 2025LiveCodeBench Pro: How Do Olympiad Medalists Judge LLMs in Competitive Programming?cited by 53
  11. 2024Chinese SimpleQA: A Chinese Factuality Evaluation for Large Language Modelscited by 40
  12. 2025Physics: Benchmarking Foundation Models on University-Level Physics Problem Solvingcited by 31
  13. 2025MathConstruct: Challenging LLM Reasoning with Constructive Proofscited by 16
  14. 2025MulDimIF: A Multi-Dimensional Constraint Framework for Evaluating and Improving Instruction Following in Large Language Modelscited by 14
  15. 2024ProcBench: Benchmark for Multi-Step Reasoning and Following Procedurecited by 12
  16. 2025EIFBENCH: Extremely Complex Instruction Following Benchmark for Large Language Modelscited by 9
  17. 2025MARS-Bench: A Multi-turn Athletic Real-world Scenario Benchmark for Dialogue Evaluationcited by 4

Built on by

Not cited yet.