# GameCommBench: A Unified Benchmark and Type-Aware Evaluation for AI-Generated Game Commentary

arXiv: https://arxiv.org/abs/2610.11129 · HTML: https://mlsanity.com/paper/2610.11129/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 22 references that are in the map.
- First posted on arXiv 8 Oct 2026.

## Builds on (most cited first, 5 of 5)

- [WhisperX: Time-Accurate Speech Transcription of Long-Form Audio](https://mlsanity.com/paper/2303.00747/) (2023)
- [Improving Chess Commentaries by Combining Language Models with Symbolic Reasoning Engines](https://mlsanity.com/paper/2212.08195/) (2022)
- [ChessQA: Evaluating Large Language Models for Chess Understanding](https://mlsanity.com/paper/2510.23948/) (2025)
- [Game-MUG: Multimodal Oriented Game Situation Understanding and Commentary Generation Dataset](https://mlsanity.com/paper/2404.19175/) (2024)
- [From Multimodal Perception to Strategic Reasoning: A Survey on AI-Generated Game Commentary](https://mlsanity.com/paper/2506.17294/) (2025)

## Abstract (arXiv)

Game commentary is an open-ended generation task requiring multimodal perception, strategic reasoning, and contextual knowledge. Existing AI-Generated Game Commentary (AI-GGC) studies remain fragmented across games, modalities, and evaluation protocols, while overlap-based or holistic evaluators fail to capture the functional heterogeneity of commentary. We introduce textscGameCommBench, a unified benchmark spanning board games, sports, and esports, with commentary aligned to heterogeneous game contexts and annotated by commentary type. We further propose Type-Aware Commentary Evaluation (TACE), a structured framework for evaluating different types of commentary. We then validate TACE for reliability and human agreement, and use it to benchmark representative AI commentators. Results reveal non-uniform capability profiles, with live observation and strategic analysis emerging as major bottlenecks. Together, textscGameCommBench and TACE provide a diagnostic foundation for comparable and interpretable AI-GGC evaluation.