Lost in Historical Time? A Polish History Matura Benchmark for Large Language Models
Not on the map. Why?
Abstract (arXiv)
Language models are widely used by students as knowledge sources, yet benchmarks rarely assess their interpretative historical reasoning. We evaluate eight leading LLMs on the Polish high school exit exam (Matura) in history - three official papers from 2023-2025, comprising short-answer questions and extended essays - and compare model performance against the human examinee population. Although models score near the ceiling, aggregate scores mask distinct competency profiles: rankings are unstable across task types, source modalities, and geographical scopes, with a consistent penalty for Polish versus Global history content. Qualitative error analysis reveals two recurring failure modes - source decontextualization, when models reason from source content rather than treating it as an object of analysis, and temporal disorientation, when responses are historically misplaced. This study introduces the first LLM history benchmark grounded in the Polish national curriculum.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 6 of its 13 references that are in the map. First posted on arXiv 3 Jun 2026.
Builds on
6 of 13 references.
References · 6
- 2020Measuring Massive Multitask Language Understandingcited by 7,963
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2024Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoningcited by 88
- 2024Time Awareness in Large Language Models: Benchmarking Fact Recall Across Timecited by 13
- 2024LLMs' Reading Comprehension Is Affected by Parametric Knowledge and Struggles with Hypothetical Statementscited by 12
- 2025LLMzSzL: a comprehensive LLM benchmark for Polishcited by 7
Built on by
Not cited yet.