mlsanityData to 7 Oct 2026

GreekBART: The First Pretrained Greek Sequence-to-Sequence Model

arXiv 2304.00869 ↗First posted 3 Apr 2023cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

The era of transfer learning has revolutionized the fields of Computer Vision and Natural Language Processing, bringing powerful pretrained models with exceptional performance across a variety of tasks. Specifically, Natural Language Processing tasks have been dominated by transformer-based language models. In Natural Language Inference and Natural Language Generation tasks, the BERT model and its variants, as well as the GPT model and its successors, demonstrated exemplary performance. However, the majority of these models are pretrained and assessed primarily for the English language or on a multilingual corpus. In this paper, we introduce GreekBART, the first Seq2Seq model based on BART-base architecture and pretrained on a large-scale Greek corpus. We evaluate and compare GreekBART against BART-random, Greek-BERT, and XLM-R on a variety of discriminative tasks. In addition, we examine its performance on two NLG tasks from GreekSUM, a newly introduced summarization dataset for the Greek language. The model, the code, and the new summarization dataset will be publicly available.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 28 of its 44 references that are in the map. First posted on arXiv 3 Apr 2023.

Builds on

28 of 44 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  7. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  8. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  9. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  10. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  11. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  12. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
16 more
  1. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  2. 2017Mixed Precision Trainingcited by 1,135
  3. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  4. 2015Teaching Machines to Read and Comprehendcited by 1,039
  5. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  6. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 913
  7. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 544
  8. 2019CamemBERT: a Tasty French Language Modelcited by 368
  9. 2020Multilingual Translation with Extensible Multilingual Pretraining and Finetuningcited by 287
  10. 2022Towards a Cleaner Document-Oriented Multilingual Crawled Corpuscited by 143
  11. 2021The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Modelscited by 118
  12. 2020BARThez: a Skilled Pretrained French Sequence-to-Sequence Modelcited by 39
  13. 2020GREEK-BERT: The Greeks visiting Sesame Streetcited by 37
  14. 2021BARTpho: Pre-trained Sequence-to-Sequence Models for Vietnamesecited by 33
  15. 2022AraBART: a Pretrained Arabic Sequence-to-Sequence Model for Abstractive Summarizationcited by 24
  16. 2018Word Embeddings from Large-Scale Greek Web Contentcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 3; 2025: 0; 2026: 1.

8 papers.

  1. 2024How Can We Effectively Expand the Vocabulary of LLMs with 0.01GB of Target Language Text?cited by 22
  2. 2024A Survey of Large Language Models for European Languagescited by 6
  3. 2026GreekMMLU: A Native-Sourced Multitask Benchmark for Evaluating Language Models in Greekcited by 6
  4. 2023FREDSum: A Dialogue Summarization Corpus for French Political Debatescited by 3
  5. 2023Sequence-to-Sequence Spanish Pre-trained Language Modelscited by 2
  6. 2023GreekT5: A Series of Greek Sequence-to-Sequence Models for News Summarizationcited by 2
  7. 2023OYXOY: A Modern NLP Test Suite for Modern Greekcited by 1
  8. 2024A Systematic Survey of Natural Language Processing for the Greek Languagecited by 1