mlsanityData to 7 Oct 2026

Can LLM Teams Play What? Where? When?

arXiv 2605.30459 ↗First posted 28 May 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) remain limited on tasks requiring indirect reasoning, cultural knowledge, and coordinated hypothesis testing. We investigate whether team-based interaction improves LLM performance in What? Where? When? (ChGK), a quiz game designed to reward collective reasoning. We introduce three team strategies: Voting, Silent Team (the captain observes final answers), and Talkative Team (the captain observes both answers and rationales). To minimize data leakage, we evaluate these strategies on a dataset consisting of 572 ChGK questions released in 2025. Using six recent large-scale open models, we show that team-based strategies outperform single-model baselines, yielding gains of up to 20 percentage points in accuracy. The best team achieves 44.23% accuracy, and approaches human team performance on questions with available human statistics. Analysis of inter-model diversity reveals that disagreement strongly predicts lower accuracy, but explanatory communication substantially mitigates performance drops. We further examine captain behavior and find no evidence of self-preference bias; access to peer rationales improves captain judgments. Overall, LLM teams function primarily as answer selection and error-filtering mechanisms rather than generators of novel solutions. Our findings highlight the importance of interaction and suggest adaptive strategies as a promising direction for multi-agent systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 12 of its 19 references that are in the map. First posted on arXiv 28 May 2026.

Builds on

12 of 19 references.

References · 12

  1. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  2. 2017SearchQA: A New Q&A Dataset Augmented with Context from a Search Enginecited by 221
  3. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 45
  4. 2019Quizbowl: The Case for Incremental Question Answeringcited by 36
  5. 2025A Survey on Large Language Model Benchmarkscited by 32
  6. 2025The Curse of CoT: On the Limitations of Chain-of-Thought in In-Context Learningcited by 30
  7. 2025Chain-of-Thought Matters: Improving Long-Context Language Models with Reasoning Path Supervisioncited by 20
  8. 2022TAPE: Assessing Few-shot Russian Language Understandingcited by 11
  9. 2025IMPROVE: Iterative Model Pipeline Refinement and Optimization Leveraging LLM Expertscited by 7
  10. 2025From Emergence to Control: Probing and Modulating Self-Reflection in Language Modelscited by 5
  11. 2025The Law of Multi-Model Collaboration: Scaling Limits of Model Ensembling for Large Language Modelscited by 3
  12. 2025Beware of Reasoning Overconfidence: Pitfalls in the Reasoning Process for Multi-solution Taskscited by 2

Built on by

Not cited yet.