# B"uy"uk Dil Modelleri iccin TR-MMLU Benchmarki: Performans Deugerlendirmesi, Zorluklar ve .Iyilecstirme Firsatlari

arXiv: https://arxiv.org/abs/2508.13044 · HTML: https://mlsanity.com/paper/2508.13044/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 5 references that are in the map.
- First posted on arXiv 18 Aug 2025.

## Builds on (most cited first, 2 of 2)

- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [TurkishMMLU: Measuring Massive Multitask Language Understanding in Turkish](https://mlsanity.com/paper/2407.12402/) (2024)

## Built on by (most cited first, 2 of 2)

- [TurkBench: A Benchmark for Evaluating Turkish Large Language Models](https://mlsanity.com/paper/2601.07020/) (2026)
- [Turkish MMLU Pro: Traceable Option Augmentation and Its Validity Limits in Turkish Multiple-Choice Evaluation](https://mlsanity.com/paper/2609.15467/) (2026)

## Abstract (arXiv)

Language models have made significant advancements in understanding and generating human language, achieving remarkable success in various applications. However, evaluating these models remains a challenge, particularly for resource-limited languages like Turkish. To address this issue, we introduce the Turkish MMLU (TR-MMLU) benchmark, a comprehensive evaluation framework designed to assess the linguistic and conceptual capabilities of large language models (LLMs) in Turkish. TR-MMLU is based on a meticulously curated dataset comprising 6,200 multiple-choice questions across 62 sections within the Turkish education system. This benchmark provides a standard framework for Turkish NLP research, enabling detailed analyses of LLMs' capabilities in processing Turkish text. In this study, we evaluated state-of-the-art LLMs on TR-MMLU, highlighting areas for improvement in model design. TR-MMLU sets a new standard for advancing Turkish NLP research and inspiring future innovations.