# The Impact of Copyrighted Material on Large Language Models: A Norwegian Perspective

arXiv: https://arxiv.org/abs/2412.09460 · HTML: https://mlsanity.com/paper/2412.09460/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 42 references that are in the map.
- First posted on arXiv 12 Dec 2024.

## Builds on (most cited first, 27 of 27)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale](https://mlsanity.com/paper/2406.17557/) (2024)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [OLMo: Accelerating the Science of Language Models](https://mlsanity.com/paper/2402.00838/) (2024)
- [CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data](https://mlsanity.com/paper/1911.00359/) (2019)
- [The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants](https://mlsanity.com/paper/2308.16884/) (2023)
- [Emerging Cross-lingual Structure in Pretrained Language Models](https://mlsanity.com/paper/1911.01464/) (2019)
- [CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages](https://mlsanity.com/paper/2309.09400/) (2023)
- [Benchmark Data Contamination of Large Language Models: A Survey](https://mlsanity.com/paper/2406.04244/) (2024)
- [The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MT](https://mlsanity.com/paper/2010.06354/) (2020)
- [A New Massive Multilingual Dataset for High-Performance Language Technologies](https://mlsanity.com/paper/2403.14009/) (2024)
- [BERTIN: Efficient Pre-Training of a Spanish Language Model using Perplexity Sampling](https://mlsanity.com/paper/2207.06814/) (2022)
- [A Fine-Grained Sentiment Dataset for Norwegian](https://mlsanity.com/paper/1911.12722/) (2019)
- [Operationalizing a National Digital Library: The Case for a Norwegian Transformer Model](https://mlsanity.com/paper/2104.09617/) (2021)
- [NorBench -- A Benchmark for Norwegian Language Models](https://mlsanity.com/paper/2305.03880/) (2023)
- [Large-Scale Contextualised Language Modelling for Norwegian](https://mlsanity.com/paper/2104.06546/) (2021)
- [Towards Data Contamination Detection for Modern Large Language Models: Limitations, Inconsistencies, and Oracle Challenges](https://mlsanity.com/paper/2409.09927/) (2024)
- [NoReC: The Norwegian Review Corpus](https://mlsanity.com/paper/1710.05370/) (2017)
- [Small Languages, Big Models: A Study of Continual Training on Languages of Norway](https://mlsanity.com/paper/2412.06484/) (2024)
- [Does Corpus Quality Really Matter for Low-Resource Languages?](https://mlsanity.com/paper/2203.08111/) (2022)
- [NLEBench+NorGLM: A Comprehensive Empirical Analysis and Benchmark Dataset for Generative Language Models in Norwegian](https://mlsanity.com/paper/2312.01314/) (2023)
- [NorQuAD: Norwegian Question Answering Dataset](https://mlsanity.com/paper/2305.01957/) (2023)

## Built on by (most cited first, 8 of 8)

- [NorEval: A Norwegian Language Understanding and Generation Evaluation Benchmark](https://mlsanity.com/paper/2504.07749/) (2025)
- [Register Always Matters: Analysis of LLM Pretraining Data Through the Lens of Language Variation](https://mlsanity.com/paper/2504.01542/) (2025)
- [A Collection of Question Answering Datasets for Norwegian](https://mlsanity.com/paper/2501.11128/) (2025)
- [If I Could Turn Back Time: Temporal Reframing as a Historical Reasoning Task for LLMs](https://mlsanity.com/paper/2511.04432/) (2025)
- [Measuring Task-Agnostic Training Data Influence Across Language Model Pretraining](https://mlsanity.com/paper/2608.13515/) (2026)
- [Fluent Alignment with Disfluent Judges: Post-training for Lower-resource Languages](https://mlsanity.com/paper/2512.08777/) (2025)
- [NorwAI's Large Language Models: Technical Report](https://mlsanity.com/paper/2601.03034/) (2026)
- [Reading the News: Adapting Large Language Models to Swedish Journalism Through Continued Pre-Training](https://mlsanity.com/paper/2608.30609/) (2026)

## Abstract (arXiv)

The use of copyrighted materials in training language models raises critical legal and ethical questions. This paper presents a framework for and the results of empirically assessing the impact of publisher-controlled copyrighted corpora on the performance of generative large language models (LLMs) for Norwegian. When evaluated on a diverse set of tasks, we found that adding both books and newspapers to the data mixture of LLMs tend to improve their performance, while the addition of fiction works seems to be detrimental. Our experiments could inform the creation of a compensation scheme for authors whose works contribute to AI development.