mlsanityData to 7 Oct 2026

Large-Scale Contextualised Language Modelling for Norwegian

arXiv 2104.06546 ↗First posted 13 Apr 2021cs.CLcited by 21

Not on the map. Why?

Abstract (arXiv)

We present the ongoing NorLM initiative to support the creation and use of very large contextualised language models for Norwegian (and in principle other Nordic languages), including a ready-to-use software environment, as well as an experience report for data preparation and training. This paper introduces the first large-scale monolingual language models for Norwegian, based on both the ELMo and BERT frameworks. In addition to detailing the training process, we present contrastive benchmark results on a suite of NLP tasks for Norwegian. For additional background and access to the data, models, and software, please see http://norlm.nlpl.eu

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 18 of its 30 references that are in the map. First posted on arXiv 13 Apr 2021.

Builds on

18 of 30 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2018Deep contextualized word representationscited by 2,285
  3. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 719
  4. 2019Large Batch Optimization for Deep Learning: Training BERT in 76 minutescited by 671
  5. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
  6. 2019To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Taskscited by 167
  7. 2020PhoBERT: Pre-trained language models for Vietnamesecited by 148
  8. 2019Multilingual is not enough: BERT for Finnishcited by 118
  9. 2019BERTje: A Dutch BERT Modelcited by 109
  10. 2020ParsBERT: Transformer-based Model for Persian Language Understandingcited by 92
  11. 2018Simpler but More Accurate Semantic Dependency Parsingcited by 46
  12. 2020GREEK-BERT: The Greeks visiting Sesame Streetcited by 37
6 more
  1. 2020Playing with Words at the National Library of Sweden -- Making a Swedish BERTcited by 35
  2. 2019A Fine-Grained Sentiment Dataset for Norwegiancited by 23
  3. 2018Towards Better UD Parsing: Deep Contextualized Word Embeddings, Ensemble, and Treebank Concatenationcited by 21
  4. 2019NorNE: Annotating Named Entities for Norwegiancited by 18
  5. 2017NoReC: The Norwegian Review Corpuscited by 17
  6. 2019High Quality ELMo Embeddings for Seven Less-Resourced Languagescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 1; 2023: 11; 2024: 4; 2025: 2; 2026: 1.

21 papers.

Sort:

  1. 2023A Family of Pretrained Transformer Language Models for Russiancited by 27
  2. 2021Operationalizing a National Digital Library: The Case for a Norwegian Transformer Modelcited by 23
  3. 2023NorBench -- A Benchmark for Norwegian Language Modelscited by 23
  4. 2023Does Manipulating Tokenization Aid Cross-Lingual Transfer? A Study on POS Tagging for Non-Standardized Languagescited by 19
  5. 2021skweak: Weak Supervision Made Easy for NLPcited by 17
  6. 2023ScandEval: A Benchmark for Scandinavian Natural Language Processingcited by 16
  7. 2024Small Languages, Big Models: A Study of Continual Training on Languages of Norwaycited by 12
  8. 2023NoCoLA: The Norwegian Corpus of Linguistic Acceptabilitycited by 9
  9. 2023NLEBench+NorGLM: A Comprehensive Empirical Analysis and Benchmark Dataset for Generative Language Models in Norwegiancited by 9
  10. 2024The Impact of Copyrighted Material on Large Language Models: A Norwegian Perspectivecited by 8
  11. 2022Measuring Harmful Representations in Scandinavian Language Modelscited by 7
  12. 2023NorQuAD: Norwegian Question Answering Datasetcited by 7
9 more
  1. 2024PersonalSum: A User-Subjective Guided Personalized Summarization Dataset for Large Language Modelscited by 5
  2. 2024Development of Pre-Trained Transformer-based Models for the Nepali Languagecited by 5
  3. 2023Measuring Normative and Descriptive Biases in Language Models Using Census Datacited by 3
  4. 2023Adversarial Capsule Networks for Romanian Satire Detection and Sentiment Analysiscited by 3
  5. 2023BRENT: Bidirectional Retrieval Enhanced Norwegian Transformercited by 2
  6. 2025Mixed Feelings: Cross-Domain Sentiment Classification of Patient Feedbackcited by 1
  7. 2025English K_Quantization of LLMs Does Not Disproportionately Diminish Multilingual Performancecited by 1
  8. 2023Aligning the Norwegian UD Treebank with Entity and Coreference Informationcited by 0
  9. 2026NorwAI's Large Language Models: Technical Reportcited by 0