Can LLM Teams Play What? Where? When?
Not on the map. Why?
Abstract (arXiv)
Large language models (LLMs) remain limited on tasks requiring indirect reasoning, cultural knowledge, and coordinated hypothesis testing. We investigate whether team-based interaction improves LLM performance in What? Where? When? (ChGK), a quiz game designed to reward collective reasoning. We introduce three team strategies: Voting, Silent Team (the captain observes final answers), and Talkative Team (the captain observes both answers and rationales). To minimize data leakage, we evaluate these strategies on a dataset consisting of 572 ChGK questions released in 2025. Using six recent large-scale open models, we show that team-based strategies outperform single-model baselines, yielding gains of up to 20 percentage points in accuracy. The best team achieves 44.23% accuracy, and approaches human team performance on questions with available human statistics. Analysis of inter-model diversity reveals that disagreement strongly predicts lower accuracy, but explanatory communication substantially mitigates performance drops. We further examine captain behavior and find no evidence of self-preference bias; access to peer rationales improves captain judgments. Overall, LLM teams function primarily as answer selection and error-filtering mechanisms rather than generators of novel solutions. Our findings highlight the importance of interaction and suggest adaptive strategies as a promising direction for multi-agent systems.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 12 of its 19 references that are in the map. First posted on arXiv 28 May 2026.
Builds on
12 of 19 references.
References · 12
- 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
- 2017SearchQA: A New Q&A Dataset Augmented with Context from a Search Enginecited by 221
- 2025Large Language Models Hallucination: A Comprehensive Surveycited by 45
- 2019Quizbowl: The Case for Incremental Question Answeringcited by 36
- 2025A Survey on Large Language Model Benchmarkscited by 32
- 2025The Curse of CoT: On the Limitations of Chain-of-Thought in In-Context Learningcited by 30
- 2025Chain-of-Thought Matters: Improving Long-Context Language Models with Reasoning Path Supervisioncited by 20
- 2022TAPE: Assessing Few-shot Russian Language Understandingcited by 11
- 2025IMPROVE: Iterative Model Pipeline Refinement and Optimization Leveraging LLM Expertscited by 7
- 2025From Emergence to Control: Probing and Modulating Self-Reflection in Language Modelscited by 5
- 2025The Law of Multi-Model Collaboration: Scaling Limits of Model Ensembling for Large Language Modelscited by 3
- 2025Beware of Reasoning Overconfidence: Pitfalls in the Reasoning Process for Multi-solution Taskscited by 2
Built on by
Not cited yet.