mlsanityData to 14 Oct 2026

RobeCzech: Czech RoBERTa, a monolingual contextualized language representation model

arXiv 2105.11314 ↗First posted 24 May 2021cs.CLcited by 33

Not on the map. Why?

Abstract (arXiv)

We present RobeCzech, a monolingual RoBERTa language representation model trained on Czech data. RoBERTa is a robustly optimized Transformer-based pretraining approach. We show that RobeCzech considerably outperforms equally-sized multilingual and Czech-trained contextualized language representation models, surpasses current state of the art in all five evaluated NLP tasks and reaches state-of-the-art results in four of them. The RobeCzech model is released publicly at https://hdl.handle.net/11234/1-3691 and https://huggingface.co/ufal/robeczech-base.

Key facts. Cited by 33 papers in the map (data through 14 Oct 2026). Builds on 19 of its 47 references that are in the map. First posted on arXiv 24 May 2021.

Builds on

19 of 47 references.

References · 19

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2016Layer Normalizationcited by 5,669
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  6. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,474
  7. 2016Enriching Word Vectors with Subword Informationcited by 1,583
  8. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,304
  9. 2019CamemBERT: a Tasty French Language Modelcited by 368
  10. 2016Deep Biaffine Attention for Neural Dependency Parsingcited by 334
  11. 2019Multilingual is not enough: BERT for Finnishcited by 119
  12. 201975 Languages, 1 Model: Parsing Universal Dependencies Universallycited by 90
7 more
  1. 2019Neural Architectures for Nested NER through Linearizationcited by 56
  2. 2015Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representationcited by 28
  3. 2021Czert -- Czech BERT-like Model for Language Representationcited by 22
  4. 2020'UFAL at MRP 2020: Permutation-invariant Semantic Parsing in PERINcited by 12
  5. 2018LemmaTag: Jointly Tagging and Lemmatizing for Morphologically-Rich Languages with BRNNscited by 8
  6. 2019Czech Text Processing with Contextual Embeddings: POS Tagging, Lemmatization, Parsing and NERcited by 3
  7. 2020HUJI-KU at MRP~2020: Two Transition-based Neural Parserscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 10; 2023: 6; 2024: 4; 2025: 5; 2026: 4.

33 papers.

Sort:

By area:

  1. 2021AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processingcited by 131
  2. 2022L3Cube-MahaCorpus and MahaBERT: Marathi Monolingual Corpus, Marathi BERT Language Models, and Resourcescited by 51
  3. 2022Speaking Multiple Languages Affects the Moral Bias of Language Modelscited by 34
  4. 2022Mono vs Multilingual BERT for Hate Speech Detection and Text Classification: A Case Study in Marathicited by 16
  5. 2022Do Multilingual Language Models Capture Differing Moral Norms?cited by 15
  6. 2023L3Cube-IndicSBERT: A simple approach for learning cross-lingual sentence representations using multilingual BERTcited by 15
  7. 2021Comparison of Czech Transformers on Text Classification Taskscited by 10
  8. 2022L3Cube-MahaSBERT and HindSBERT: Sentence BERT Models and Benchmarking BERT Sentence Representations for Hindi and Marathicited by 10
  9. 2022'UFAL CorPipe at CRAC 2022: Effectivity of Multilingual Models for Coreference Resolutioncited by 8
  10. 2023Quality and Efficiency of Manual Annotation: Pre-annotation Biascited by 8
  11. 2021Siamese BERT-based Model for Web Search Relevance Ranking Evaluated on a New Czech Datasetcited by 7
  12. 2021Training dataset and dictionary sizes matter in BERT models: the case of Baltic languagescited by 7
21 more
  1. 2022CsFEVER and CTKFacts: Acquiring Czech data for fact verificationcited by 6
  2. 2024A Survey of Large Language Models for European Languagescited by 6
  3. 2022Czech Dataset for Cross-lingual Subjectivity Classificationcited by 5
  4. 2023Pipeline and Dataset Generation for Automated Fact-checking in Almost Any Languagecited by 4
  5. 2025Czech Dataset for Complex Aspect-Based Sentiment Analysis Taskscited by 4
  6. 2023Some Like It Small: Czech Semantic Embedding Models for Industry Applicationscited by 3
  7. 2025Unpacking Robustness in Inflectional Languages: Adversarial Evaluation and Mechanistic Insightscited by 3
  8. 2022Unsupervised extraction, labelling and clustering of segments from clinical notescited by 2
  9. 2023A Dataset and Strong Baselines for Classification of Czech News Textscited by 2
  10. 2024How Gender Interacts with Political Values: A Case Study on Czech BERT Modelscited by 2
  11. 2024Open-Source Web Service with Morphological Dictionary-Supplemented Deep Learning for Morphosyntactic Analysis of Czechcited by 2
  12. 2024GPT Czech Poet: Generation of Czech Poetic Strophes with Language Modelscited by 2
  13. 2025Prompt-Based Approach for Czech Sentiment Analysiscited by 2
  14. 2026Different Time, Different Language: Revisiting the Bias Against Non-Native Speakers in GPT Detectorscited by 2
  15. 2022BERTuit: Understanding Spanish language in Twitter through a native transformercited by 1
  16. 2023AlbNER: A Corpus for Named Entity Recognition in Albaniancited by 1
  17. 2025L3Cube-MahaSTS: A Marathi Sentence Similarity Dataset and Modelscited by 1
  18. 2025Intrinsic vs. Extrinsic Evaluation of Czech Sentence Embeddings: Semantic Relevance Doesn't Help with MT Evaluationcited by 0
  19. 2026Glitter: Visualizing Lexical Surprisal for Readability in Administrative Textscited by 0
  20. 2026CzechTopic: A Benchmark for Zero-Shot Topic Localization in Historical Czech Documentscited by 0
  21. 2026Zero-Shot to Full-Resource: Cross-lingual Transfer Strategies for Aspect-Based Sentiment Analysiscited by 0