# Lost in Historical Time? A Polish History Matura Benchmark for Large Language Models

arXiv: https://arxiv.org/abs/2608.12343 · HTML: https://mlsanity.com/paper/2608.12343/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 13 references that are in the map.
- First posted on arXiv 3 Jun 2026.

## Builds on (most cited first, 6 of 6)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Test of Time: A Benchmark for Evaluating LLMs on Temporal Reasoning](https://mlsanity.com/paper/2406.09170/) (2024)
- [Time Awareness in Large Language Models: Benchmarking Fact Recall Across Time](https://mlsanity.com/paper/2409.13338/) (2024)
- [LLMs' Reading Comprehension Is Affected by Parametric Knowledge and Struggles with Hypothetical Statements](https://mlsanity.com/paper/2404.06283/) (2024)
- [LLMzSzL: a comprehensive LLM benchmark for Polish](https://mlsanity.com/paper/2501.02266/) (2025)

## Abstract (arXiv)

Language models are widely used by students as knowledge sources, yet benchmarks rarely assess their interpretative historical reasoning. We evaluate eight leading LLMs on the Polish high school exit exam (Matura) in history - three official papers from 2023-2025, comprising short-answer questions and extended essays - and compare model performance against the human examinee population. Although models score near the ceiling, aggregate scores mask distinct competency profiles: rankings are unstable across task types, source modalities, and geographical scopes, with a consistent penalty for Polish versus Global history content. Qualitative error analysis reveals two recurring failure modes - source decontextualization, when models reason from source content rather than treating it as an object of analysis, and temporal disorientation, when responses are historically misplaced. This study introduces the first LLM history benchmark grounded in the Polish national curriculum.