mlsanityData to 14 Oct 2026

RigoBERTa: A State-of-the-Art Language Model For Spanish

arXiv 2205.10233 ↗First posted 27 Apr 2022cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

This paper presents RigoBERTa, a State-of-the-Art Language Model for Spanish. RigoBERTa is trained over a well-curated corpus formed up from different subcorpora with key features. It follows the DeBERTa architecture, which has several advantages over other architectures of similar size as BERT or RoBERTa. RigoBERTa performance is assessed over 13 NLU tasks in comparison with other available Spanish language models, namely, MarIA, BERTIN and BETO. RigoBERTa outperformed the three models in 10 out of the 13 tasks, achieving new "State-of-the-Art" results.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 30 of its 47 references that are in the map. First posted on arXiv 27 Apr 2022.

Builds on

30 of 47 references.

References · 30

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  7. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  8. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  9. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
  10. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  11. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  12. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
18 more
  1. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,475
  2. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
  3. 2019On the Cross-lingual Transferability of Monolingual Representationscited by 578
  4. 2021Datasets: A Community Library for Natural Language Processingcited by 530
  5. 2019PAWS: Paraphrase Adversaries from Word Scramblingcited by 402
  6. 2019CamemBERT: a Tasty French Language Modelcited by 368
  7. 2019PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identificationcited by 264
  8. 2019FlauBERT: Unsupervised Language Model Pre-training for Frenchcited by 153
  9. 2020German's Next Language Modelcited by 144
  10. 2020A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languagescited by 131
  11. 2020RobBERT: a Dutch RoBERTa-based Language Modelcited by 88
  12. 2019Macro F1 and Macro F1cited by 70
  13. 2020GottBERT: a pure German Language Modelcited by 49
  14. 2018A Corpus for Multilingual Document Classification in Eight Languagescited by 42
  15. 2019Automatic Spanish Translation of the SQuAD Dataset for Multilingual Question Answeringcited by 35
  16. 2021Biomedical and Clinical Language Models for Spanish: On the Benefits of Domain-Specific Pretraining in a Mid-Resource Scenariocited by 11
  17. 2017Correction of "A Comparative Study to Benchmark Cross-project Defect Prediction Approaches"cited by 4
  18. 2020Overview of CAPITEL Shared Tasks at IberLEF 2020: Named Entity Recognition and Universal Dependencies Parsingcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 1; 2025: 0; 2026: 1.

5 papers.

  1. 2023Leveraging Large Language Models for Topic Classification in the Domain of Public Affairscited by 13
  2. 2024An evaluation of LLM code generation capabilities through graded exercisescited by 3
  3. 2023Sequence-to-Sequence Spanish Pre-trained Language Modelscited by 2
  4. 2023A Survey of Spanish Clinical Language Modelscited by 0
  5. 2026MrBERT: Modern Multilingual Encoders via Vocabulary, Domain, and Dimensional Adaptationcited by 0