mlsanityData to 7 Oct 2026

AraBART: a Pretrained Arabic Sequence-to-Sequence Model for Abstractive Summarization

arXiv 2203.10945 ↗First posted 21 Mar 2022cs.CLcited by 24

Not on the map. Why?

Abstract (arXiv)

Like most natural language understanding and generation tasks, state-of-the-art models for summarization are transformer-based sequence-to-sequence architectures that are pretrained on large corpora. While most existing models focused on English, Arabic remained understudied. In this paper we propose AraBART, the first Arabic model in which the encoder and the decoder are pretrained end-to-end, based on BART. We show that AraBART achieves the best performance on multiple abstractive summarization datasets, outperforming strong baselines including a pretrained Arabic BERT-based model and multilingual mBART and mT5 models.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 29 of its 55 references that are in the map. First posted on arXiv 21 Mar 2022.

Builds on

29 of 55 references.

References · 29

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  6. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  7. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  8. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  9. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  10. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  11. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  12. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
17 more
  1. 2019Text Summarization with Pretrained Encoderscited by 553
  2. 2015A Neural Attention Model for Abstractive Sentence Summarizationcited by 394
  3. 2020AraBERT: Transformer-based Model for Arabic Language Understandingcited by 359
  4. 2021XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languagescited by 346
  5. 2016SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documentscited by 245
  6. 2019Leveraging Pre-trained Checkpoints for Sequence Generation Taskscited by 182
  7. 2020GSum: A General Framework for Guided Neural Abstractive Summarizationcited by 129
  8. 2018Ranking Sentences for Extractive Summarization with Reinforcement Learningcited by 128
  9. 2021The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Modelscited by 118
  10. 2021CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generationcited by 81
  11. 2021Improving Faithfulness in Abstractive Summarization with Contrast Candidate Generation and Selectioncited by 75
  12. 2015LCSTS: A Large Scale Chinese Short Text Summarization Datasetcited by 65
  13. 2018Neural Document Summarization by Jointly Learning to Score and Select Sentencescited by 56
  14. 2020BARThez: a Skilled Pretrained French Sequence-to-Sequence Modelcited by 39
  15. 2020Pretrained Language Model Embryology: The Birth of ALBERTcited by 30
  16. 2021FrugalScore: Learning Cheaper, Lighter and Faster Evaluation Metricsfor Automatic Text Generationcited by 19
  17. 2020BERT Fine-tuning For Arabic Text Summarizationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 10; 2025: 3; 2026: 1.

24 papers.

Sort:

By area:

  1. 2024ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabiccited by 87
  2. 2023Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Modelscited by 68
  3. 2024ALLaM: Large Language Models for Arabic and Englishcited by 55
  4. 2023NADI 2023: The Fourth Nuanced Arabic Dialect Identification Shared Taskcited by 29
  5. 2024NADI 2024: The Fifth Nuanced Arabic Dialect Identification Shared Taskcited by 23
  6. 2023Advancements in Arabic Grammatical Error Detection and Correction: An Empirical Investigationcited by 22
  7. 2023Dolphin: A Challenging and Diverse Benchmark for Arabic NLGcited by 22
  8. 2024BiMediX: Bilingual Medical Mixture of Experts LLMcited by 21
  9. 2024A Survey of Large Language Models for Arabic Language and its Dialectscited by 13
  10. 2023Beyond English: Evaluating LLMs for Arabic Grammatical Error Correctioncited by 12
  11. 2023GreekBART: The First Pretrained Greek Sequence-to-Sequence Modelcited by 8
  12. 2023AraMUS: Pushing the Limits of Data and Model Scale for Arabic Natural Language Processingcited by 7
12 more
  1. 2024Exploiting Dialect Identification in Automatic Dialectal Text Normalizationcited by 7
  2. 2023Octopus: A Multitask Model and Toolkit for Arabic Natural Language Generationcited by 6
  3. 2023ChatGPT for Arabic Grammatical Error Correctioncited by 5
  4. 2024Improving Language Models Trained on Translated Data with Continual Pre-Training and Dictionary Learning Analysiscited by 5
  5. 2024EgyBERT: A Large Language Model Pretrained on Egyptian Dialect Corporacited by 5
  6. 2024Transformer Models in Education: Summarizing Science Textbooks with AraBART, MT5, AraT5, and mBARTcited by 3
  7. 2023Sequence-to-Sequence Spanish Pre-trained Language Modelscited by 2
  8. 2024FASSILA: A Corpus for Algerian Dialect Fake News Detection and Sentiment Analysiscited by 2
  9. 2025A Tale of Two Scripts: Transliteration and Post-Correction for Judeo-Arabiccited by 1
  10. 2025AraFinNews: Arabic Financial Summarisation with Domain-Adapted LLMscited by 1
  11. 2026Multi-Task Instruction Tuning via Data Scheduling for Low-Resource Arabic SpeechLLMscited by 1
  12. 2025AraToken: Optimizing Arabic Tokenization with Normalization Pipeline and Language Extension for Qwen3cited by 0