# Automated essay scoring in Arabic: a dataset and analysis of a BERT-based system

arXiv: https://arxiv.org/abs/2407.11212 · HTML: https://mlsanity.com/paper/2407.11212/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 41 references that are in the map.
- First posted on arXiv 15 Jul 2024.

## Builds on (most cited first, 5 of 5)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [AraBERT: Transformer-based Model for Arabic Language Understanding](https://mlsanity.com/paper/2003.00104/) (2020)
- [AraGPT2: Pre-Trained Transformer for Arabic Language Generation](https://mlsanity.com/paper/2012.15520/) (2020)
- [On the Use of BERT for Automated Essay Scoring: Joint Learning of Multi-Scale Essay Representation](https://mlsanity.com/paper/2205.03835/) (2022)
- [An Automated System for Essay Scoring of Online Exams in Arabic based on Stemming Techniques and Levenshtein Edit Operations](https://mlsanity.com/paper/1611.02815/) (2016)

## Built on by (most cited first, 5 of 5)

- [Enhancing Arabic Automated Essay Scoring with Synthetic Data and Error Injection](https://mlsanity.com/paper/2503.17739/) (2025)
- [LAILA: A Large Trait-Based Dataset for Arabic Automated Essay Scoring](https://mlsanity.com/paper/2512.24235/) (2025)
- [Qayyem: A Real-time Platform for Scoring Proficiency of Arabic Essays](https://mlsanity.com/paper/2603.01009/) (2026)
- [Structured Prompting for Arabic Essay Proficiency: A Trait-Centric Evaluation Approach](https://mlsanity.com/paper/2603.19668/) (2026)
- [Automated Scoring of Arabic Text Using Large Language Models: A Literature Review](https://mlsanity.com/paper/2606.09830/) (2026)

## Abstract (arXiv)

Automated Essay Scoring (AES) holds significant promise in the field of education, helping educators to mark larger volumes of essays and provide timely feedback. However, Arabic AES research has been limited by the lack of publicly available essay data. This study introduces AR-AES, an Arabic AES benchmark dataset comprising 2046 undergraduate essays, including gender information, scores, and transparent rubric-based evaluation guidelines, providing comprehensive insights into the scoring process. These essays come from four diverse courses, covering both traditional and online exams. Additionally, we pioneer the use of AraBERT for AES, exploring its performance on different question types. We find encouraging results, particularly for Environmental Chemistry and source-dependent essay questions. For the first time, we examine the scale of errors made by a BERT-based AES system, observing that 96.15 percent of the errors are within one point of the first human marker's prediction, on a scale of one to five, with 79.49 percent of predictions matching exactly. In contrast, additional human markers did not exceed 30 percent exact matches with the first marker, with 62.9 percent within one mark. These findings highlight the subjectivity inherent in essay grading, and underscore the potential for current AES technology to assist human markers to grade consistently across large classes.