# The Growing Gains and Pains of Iterative Web Corpora Crawling: Insights from South Slavic CLASSLA-web 2.0 Corpora

arXiv: https://arxiv.org/abs/2601.11170 · HTML: https://mlsanity.com/paper/2601.11170/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 29 references that are in the map.
- First posted on arXiv 16 Jan 2026.

## Builds on (most cited first, 10 of 10)

- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language](https://mlsanity.com/paper/2506.20920/) (2025)
- [A New Massive Multilingual Dataset for High-Performance Language Technologies](https://mlsanity.com/paper/2403.14009/) (2024)
- [LLM Teacher-Student Framework for Text Classification With No Manually Annotated Data: A Case Study in IPTC News Topic Classification](https://mlsanity.com/paper/2411.19638/) (2024)
- [Generative Model for Less-Resourced Language with 1 billion parameters](https://mlsanity.com/paper/2410.06898/) (2024)
- [Do Language Models Care About Text Quality? Evaluating Web-Crawled Corpora Across 11 Languages](https://mlsanity.com/paper/2403.08693/) (2024)
- [CLASSLA-web: Comparable Web Corpora of South Slavic Languages Enriched with Linguistic and Genre Annotation](https://mlsanity.com/paper/2403.12721/) (2024)
- [Language Models on a Diet: Cost-Efficient Development of Encoders for Closely-Related Languages via Additional Pretraining](https://mlsanity.com/paper/2404.05428/) (2024)
- [CLASSLA-Express: a Train of CLARIN.SI Workshops on Language Resources and Tools with Easily Expanding Route](https://mlsanity.com/paper/2412.01386/) (2024)

## Abstract (arXiv)

Crawling national top-level domains has proven to be highly effective for collecting texts in less-resourced languages. This approach has been recently used for South Slavic languages and resulted in the largest general corpora for this language group: the CLASSLA-web 1.0 corpora. Building on this success, we established a continuous crawling infrastructure for iterative national top-level domain crawling across South Slavic and related webs. We present the first outcome of this crawling infrastructure - the CLASSLA-web 2.0 corpus collection, with substantially larger web corpora containing 17.0 billion words in 38.1 million texts in seven languages: Bosnian, Bulgarian, Croatian, Macedonian, Montenegrin, Serbian, and Slovenian. In addition to genre categories, the new version is also automatically annotated with topic labels. Comparing CLASSLA-web 2.0 with its predecessor reveals that only one-fifth of the texts overlap, showing that re-crawling after just two years yields largely new content. However, while the new web crawls bring growing gains, we also notice growing pains - a manual inspection of top domains reveals a visible degradation of web content, as machine-generated sites now contribute a significant portion of texts.