mlsanityData to 7 Oct 2026

Language Resources for Dutch Large Language Modelling

arXiv 2312.12852 ↗First posted 20 Dec 2023cs.CL cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

Despite the rapid expansion of types of large language models, there remains a notable gap in models specifically designed for the Dutch language. This gap is not only a shortage in terms of pretrained Dutch models but also in terms of data, and benchmarks and leaderboards. This work provides a small step to improve the situation. First, we introduce two fine-tuned variants of the Llama 2 13B model. We first fine-tuned Llama 2 using Dutch-specific web-crawled data and subsequently refined this model further on multiple synthetic instruction and chat datasets. These datasets as well as the model weights are made available. In addition, we provide a leaderboard to keep track of the performance of (Dutch) models on a number of generation tasks, and we include results of a number of state-of-the-art models, including our own. Finally we provide a critical conclusion on what we believe is needed to push forward Dutch language models and the whole eco-system around the models.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 13 of its 23 references that are in the map. First posted on arXiv 20 Dec 2023.

Builds on

13 of 23 references.

References · 13

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  7. 2017Deep reinforcement learning from human preferencescited by 3,846
  8. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  9. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  10. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  11. 2023Zephyr: Direct Distillation of LM Alignmentcited by 498
  12. 2023Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedbackcited by 189
1 more
  1. 2023DUMB: A Benchmark for Smart Evaluation of Dutch Modelscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 2; 2026: 1.

11 papers.

  1. 2025Leveraging Open-Source Large Language Models for Clinical Information Extraction in Resource-Constrained Settingscited by 7
  2. 2024From Bytes to Borsch: Fine-Tuning Gemma and Mistral for the Ukrainian Language Representationcited by 6
  3. 2024A Survey of Large Language Models for European Languagescited by 6
  4. 2024BEIR-NL: Zero-shot Information Retrieval Benchmark for the Dutch Languagecited by 6
  5. 2024ChocoLlama: Lessons Learned From Teaching Llamas Dutchcited by 5
  6. 2024GEITje 7B Ultra: A Conversational Model for Dutchcited by 4
  7. 2024A Dutch Financial Large Language Modelcited by 2
  8. 2024The advantages of context specific language models: the case of the Erasmian Language Modelcited by 1
  9. 2024Is Our Chatbot Telling Lies? Assessing Correctness of an LLM-based Dutch Support Chatbotcited by 0
  10. 2025Interpretable phenotyping of Heart Failure patients with Dutch discharge letterscited by 0
  11. 2026When Context Misleads: Surprisal, Energy and Attention Entropy as Metrics of Coherence Illusions in LLMscited by 0