mlsanityData to 7 Oct 2026

AraELECTRA: Pre-Training Text Discriminators for Arabic Language Understanding

arXiv 2012.15516 ↗First posted 31 Dec 2020cs.CLcited by 43

Not on the map. Why?

Abstract (arXiv)

Advances in English language representation enabled a more sample-efficient pre-training task by Efficiently Learning an Encoder that Classifies Token Replacements Accurately (ELECTRA). Which, instead of training a model to recover masked tokens, it trains a discriminator model to distinguish true input tokens from corrupted tokens that were replaced by a generator network. On the other hand, current Arabic language representation approaches rely only on pretraining via masked language modeling. In this paper, we develop an Arabic language representation model, which we name AraELECTRA. Our model is pretrained using the replaced token detection objective on large Arabic text corpora. We evaluate our model on multiple Arabic NLP tasks, including reading comprehension, sentiment analysis, and named-entity recognition and we show that AraELECTRA outperforms current state-of-the-art Arabic language representation models, given the same pretraining data and with even a smaller model size.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 6; 2023: 9; 2024: 7; 2025: 11; 2026: 3.

43 papers.

Sort:

By area:

  1. 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 241
  2. 2021AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processingcited by 131
  3. 2024ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabiccited by 87
  4. 2023Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Modelscited by 68
  5. 2022ORCA: A Challenging Benchmark for Arabic Language Understandingcited by 46
  6. 2021NADI 2021: The Second Nuanced Arabic Dialect Identification Shared Taskcited by 32
  7. 2023NADI 2023: The Fourth Nuanced Arabic Dialect Identification Shared Taskcited by 29
  8. 2023Towards a Robust Detection of Language Model Generated Text: Is ChatGPT that Easy to Detect?cited by 23
  9. 2021Masader: Metadata Sourcing for Arabic Text and Speech Data Resourcescited by 20
  10. 2023ArAIEval Shared Task: Persuasion Techniques and Disinformation Detection in Arabic Textcited by 19
  11. 2023Large Language Models for Propaganda Span Annotationcited by 19
  12. 2024A Survey of Large Language Models for Arabic Language and its Dialectscited by 13
31 more
  1. 2023A Survey on Arabic Named Entity Recognition: Past, Recent Advances, and Future Trendscited by 8
  2. 2023AraMUS: Pushing the Limits of Data and Model Scale for Arabic Natural Language Processingcited by 7
  3. 2024ArabianGPT: Native Arabic GPT-based Large Language Modelcited by 7
  4. 2024Optimized Quran Passage Retrieval Using an Expanded QA Dataset and Fine-Tuned Language Modelscited by 5
  5. 2021AraBERT and Farasa Segmentation Based Approach For Sarcasm and Sentiment Detection in Arabic Tweetscited by 4
  6. 2022DTW at Qur'an QA 2022: Utilising Transfer Learning with Transformers for Question Answering in a Low-resource Domaincited by 4
  7. 2022Data Augmentation using Transformers and Similarity Measures for Improving Arabic Text Classificationcited by 3
  8. 2024CLEANANERCorp: Identifying and Correcting Incorrect Labels in the ANERcorp Datasetcited by 3
  9. 2025The Arabic AI Fingerprint: Stylometric Analysis and Detection of Large Language Models Textcited by 3
  10. 2023Transformers and Ensemble methods: A solution for Hate Speech Detection in Arabic languagescited by 2
  11. 2025Two-Stage Quranic QA via Ensemble Retrieval and Instruction-Tuned Answer Extractioncited by 2
  12. 2021Dialect Identification in Nuanced Arabic Tweets Using Farasa Segmentation and AraBERTcited by 1
  13. 2022Post-hoc analysis of Arabic transformer modelscited by 1
  14. 2023Detecting Suicidality in Arabic Tweets Using Machine Learning and Deep Learning Techniquescited by 1
  15. 2024Ensemble of pre-trained language models and data augmentation for hate speech detection from Arabic tweetscited by 1
  16. 2025Advancing Arabic Reverse Dictionary Systems: A Transformer-Based Approach with Dataset Construction Guidelinescited by 1
  17. 2025Enhanced Arabic Text Retrieval with Attentive Relevance Scoringcited by 1
  18. 2025AHaSIS: Shared Task on Sentiment Analysis for Arabic Dialectscited by 1
  19. 2021Pre-trained Transformer-Based Approach for Arabic Question Answering : A Comparative Studycited by 0
  20. 2022Interpreting Arabic Transformer Modelscited by 0
  21. 2022Large Language Models for Multi-label Propaganda Detectioncited by 0
  22. 2024On the importance of Data Scale in Pretraining Arabic Language Modelscited by 0
  23. 2025IntegrityAI at GenAI Detection Task 2: Detecting Machine-Generated Academic Essays in English and Arabic Using ELECTRA and Stylometrycited by 0
  24. 2025QU-NLP at CheckThat! 2025: Multilingual Subjectivity in News Articles Detection using Feature-Augmented Transformer Models with Sequential Cross-Lingual Fine-Tuningcited by 0
  25. 2025!MSA at BAREC Shared Task 2025: Ensembling Arabic Transformers for Readability Assessmentcited by 0
  26. 2025From Human Annotation to Automation: LLM-in-the-Loop Active Learning for Arabic Sentiment Analysiscited by 0
  27. 2025ALHD: A Large-Scale and Multigenre Benchmark Dataset for Arabic LLM-Generated Text Detectioncited by 0
  28. 2025BUSTED at AraGenEval Shared Task: A Comparative Study of Transformer-Based Models for Arabic AI-Generated Text Detectioncited by 0
  29. 2026MURAD: A Large-Scale Multi-Domain Unified Reverse Arabic Dictionary Datasetcited by 0
  30. 2026Echoes Across Vietnam's Highlands, Delta, and Coast: A Multilingual Corpus for Cham, Khmer, and Tay-Nungcited by 0
  31. 2026AraSSM: A bidirectional state-space encoder for Arabic masked language modelingcited by 0