mlsanityData to 14 Oct 2026

Danish Foundation Models

arXiv 2311.07264 ↗First posted 13 Nov 2023cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Large language models, sometimes referred to as foundation models, have transformed multiple fields of research. However, smaller languages risk falling behind due to high training costs and small incentives for large companies to train these models. To combat this, the Danish Foundation Models project seeks to provide and maintain open, well-documented, and high-quality foundation models for the Danish language. This is achieved through broad cooperation with public and private institutions, to ensure high data quality and applicability of the trained models. We present the motivation of the project, the current status, and future perspectives.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 17 of its 34 references that are in the map. First posted on arXiv 13 Nov 2023.

Builds on

17 of 34 references.

References · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2020Scaling Laws for Neural Language Modelscited by 6,586
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  6. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  7. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  8. 2018Datasheets for Datasetscited by 1,386
  9. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
  10. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  11. 2021Deduplicating Training Data Makes Language Models Bettercited by 714
  12. 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 613
5 more
  1. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 601
  2. 2023Extrapolating Large Language Models to Non-English by Aligning Languagescited by 69
  3. 2020ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generatorscited by 27
  4. 2020The Danish Gigaword Projectcited by 17
  5. 2022Training a T5 Using Lab-sized Resourcescited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 1.

4 papers.

  1. 2024The Scandinavian Embedding Benchmarks: Comprehensive Assessment of Multilingual and Monolingual Text Embeddingcited by 26
  2. 2025Dynaword: From One-shot to Continuously Developed Datasetscited by 3
  3. 2024Open Llama2 Model for the Lithuanian Languagecited by 2
  4. 2026Retrieving Biblical Intertextual References in Karen Blixen's Seven Gothic Talescited by 0