GPT-NL Public Corpus: A Permissively Licensed, Dutch-First Dataset for LLM Pre-training
Not on the map. Why?
Abstract (arXiv)
We present the GPT-NL Public Corpus, the biggest permissively licensed corpus of Dutch language resources. The GPT-NL Public Corpus contains 21 Dutch-only collections totalling 36B preprocessed Dutch tokens not present in any other LLM pretraining corpus. Additionally, the corpus includes roughly 207B English, 232B Code, and 48B German/Danish tokens taken from existing sets which we further curated for compliance. This corpus includes curated data from large existing corpora like Common Corpus and Common Crawl, as well as newly created Dutch-specific collections. Most newly created Dutch collections consist of content collected in collaboration with organisations or synthetically augmented content. All data is collected and evaluated with the aim of facilitating the creation of (commercial) language models that are lawful, useful and non-harmful. All data included in the GPT-NL Public Corpus is sourced from datasets with permissive licensing and is curated and redistributed under a CC-BY license. The full dataset is publicly available on the Hugging Face Hub.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 9 of its 25 references that are in the map. First posted on arXiv 1 Apr 2026.
Builds on
9 of 25 references.
References · 9
- 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
- 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 475
- 2024A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Modelscited by 266
- 2023CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languagescited by 172
- 2021MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfercited by 89
- 2025HalluLens: LLM Hallucination Benchmarkcited by 88
- 2023At Which Training Stage Does Code Data Help LLMs Reasoning?cited by 82
- 2024A New Massive Multilingual Dataset for High-Performance Language Technologiescited by 56
- 2013DGT-TM: A freely Available Translation Memory in 22 Languagescited by 16
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026Language, Language Models, and What We're Talking Aboutcited by 0
- 2026Language, Language Models, and What We're Talking About3 Sep 2026