mlsanityData to 7 Oct 2026

BERTi'c -- The Transformer Language Model for Bosnian, Croatian, Montenegrin and Serbian

arXiv 2104.09243 ↗First posted 19 Apr 2021cs.CLcited by 21

Not on the map. Why?

Abstract (arXiv)

In this paper we describe a transformer model pre-trained on 8 billion tokens of crawled text from the Croatian, Bosnian, Serbian and Montenegrin web domains. We evaluate the transformer model on the tasks of part-of-speech tagging, named-entity-recognition, geo-location prediction and commonsense causal reasoning, showing improvements on all tasks over state-of-the-art models. For commonsense reasoning evaluation, we introduce COPA-HR -- a translation of the Choice of Plausible Alternatives (COPA) dataset into Croatian. The BERTi'c model is made available for free usage and further task-specific fine-tuning through HuggingFace.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 8 of its 25 references that are in the map. First posted on arXiv 19 Apr 2021.

Builds on

8 of 25 references.

References · 8

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  5. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 719
  6. 2019CamemBERT: a Tasty French Language Modelcited by 368
  7. 2020XCOPA: A Multilingual Dataset for Causal Commonsense Reasoningcited by 330
  8. 2019BERTje: A Dutch BERT Modelcited by 109

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 3; 2023: 3; 2024: 8; 2025: 4; 2026: 1.

21 papers.

Sort:

By area:

  1. 2022Geographic Adaptation of Pretrained Language Modelscited by 14
  2. 2023NollySenti: Leveraging Transfer Learning and Machine Translation for Nigerian Movie Sentiment Classificationcited by 14
  3. 2024Multilingual transformer and BERTopic for short text topic modeling: The case of Serbiancited by 6
  4. 2021Evaluation of contextual embeddings on less-resourced languagescited by 4
  5. 2022CoRAL: a Context-aware Croatian Abusive Language Datasetcited by 4
  6. 2024Do Language Models Care About Text Quality? Evaluating Web-Crawled Corpora Across 11 Languagescited by 4
  7. 2024Language Models on a Diet: Cost-Efficient Development of Encoders for Closely-Related Languages via Additional Pretrainingcited by 4
  8. 2024LLMs for Targeted Sentiment in News Headlines: Exploring the Descriptive-Prescriptive Dilemmacited by 3
  9. 2024Data-Augmentation-Based Dialectal Adaptation for LLMscited by 3
  10. 2022The ParlaSent-BCS dataset of sentiment-annotated parliamentary debates from Bosnia-Herzegovina, Croatia, and Serbiacited by 2
  11. 2021EENLP: Cross-lingual Eastern European NLP Indexcited by 1
  12. 2024New Textual Corpora for Serbian Language Modelingcited by 1
9 more
  1. 2025Charting the European LLM Benchmarking Landscape: A New Taxonomy and a Set of Best Practicescited by 1
  2. 2026Lyapunov Spectral Analysis of Speech Embedding Trajectories in Psychosiscited by 1
  3. 2023A Survey of Resources and Methods for Natural Language Processing of Serbian Languagecited by 0
  4. 2023Croatian Film Review Dataset (Cro-FiReDa): A Sentiment Annotated Dataset of Film Reviewscited by 0
  5. 2024Novi jezivcki modeli za srpski jezikcited by 0
  6. 2024Synthetic Dataset Creation and Fine-Tuning of Transformer Models for Question Answering in Serbiancited by 0
  7. 2025Towards Open Foundation Language Model and Corpus for Macedonian: A Low-Resource Languagecited by 0
  8. 2025Characterizing Linguistic Shifts in Croatian News via Diachronic Word Embeddingscited by 0
  9. 2025What Makes You CLIC: Detection of Croatian Clickbait Headlinescited by 0