When Trivia Is Not Trivial: Everyday Knowledge Failures in Multilingual LLMs
Not on the map. Why?
Abstract (arXiv)
Quiz rooms, trivia nights, and quiz shows challenge human knowledge across a wide range of topics, from canonical facts to everyday culture. In this paper, we examine whether large language models (LLMs) can perform competitively in such settings, using quiz-style questions to test them on both common and niche topics. We introduce TriviaRoomQA, a multilingual benchmark designed to evaluate everyday, culturally grounded, and long-tail knowledge across 288 topics. The benchmark contains 3,300 parallel multiple-choice questions in six European languages and additional 5,340 French-only questions for a more fine-grained case study. We evaluate 30 open-weight LLMs from European, Asian, and North American providers, covering models from 7 to 70B parameters. We find that models are strong on knowledge-intensive topics such as history, geography, and mathematics, but substantially weaker on everyday popular-culture topics such as celebrities, music, movies, and news. Moreover, model performance varies across languages even for the same underlying questions, suggesting that access to factual knowledge is not always language-independent. In sum, our dataset and experiments demonstrate an important knowledge gap which is not captured by existing academic-based saturated benchmarks.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 49 references that are in the map. First posted on arXiv 23 Jul 2026.
Builds on
30 of 49 references.
References · 30
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
- 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
- 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
- 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
- 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
- 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 716
- 2025Humanity's Last Examcited by 491
- 2024Measuring short-form factuality in large language modelscited by 373
- 2024BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languagescited by 173
- 2023Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs?cited by 168
18 more
- 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 167
- 2024Infini-gram: Scaling Unbounded n-gram Language Models to a Trillion Tokenscited by 153
- 2025FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Languagecited by 143
- 2024Massively Multi-Cultural Knowledge Acquisition & LM Benchmarkingcited by 63
- 2025Culture is Not Trivia: Sociocultural Theory for Cultural NLPcited by 40
- 2025Break the Checkbox: Challenging Closed-Style Evaluations of Cultural Alignment in LLMscited by 26
- 2024MERA: A Comprehensive LLM Evaluation in Russiancited by 22
- 2024Beneath the Surface of Consistency: Exploring Cross-lingual Knowledge Representation Sharing in LLMscited by 21
- 2025Hire Your Anthropologist! Rethinking Culture Benchmarks Through an Anthropological Lenscited by 14
- 2025The Lucie-7B LLM and the Lucie Training Dataset: Open resources for multilingual language generationcited by 10
- 2025Culture is Everywhere: A Call for Intentionally Cultural Evaluationcited by 10
- 2024LL"aMmlein: Transparent, Compact and Competitive German-Only Language Models from Scratchcited by 9
- 2025Gaperon: A Peppered English-French Generative Language Model Suitecited by 9
- 2026Long-Tail Knowledge in Large Language Models: Taxonomy, Mechanisms, Interventions and Implicationscited by 5
- 2025LPFQA: A Long-Tail Professional Forum-based Benchmark for LLM Evaluationcited by 4
- 2026TildeOpen LLM: Leveraging Curriculum Learning to Achieve Equitable Language Representationcited by 2
- 2026Leveraging Wikidata for Geographically Informed Sociocultural Bias Dataset Creation: Application to Latin Americacited by 2
- 2026CulturALL: Benchmarking Multilingual and Multicultural Competence of LLMs on Grounded Taskscited by 1
Built on by
Not cited yet.