# Data Augmentation for Automated Essay Scoring using Transformer Models

arXiv: https://arxiv.org/abs/2210.12809 · HTML: https://mlsanity.com/paper/2210.12809/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 27 references that are in the map.
- First posted on arXiv 23 Oct 2022.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [On the Use of BERT for Automated Essay Scoring: Joint Learning of Multi-Scale Essay Representation](https://mlsanity.com/paper/2205.03835/) (2022)
- [Automated Essay Scoring Using Transformer Models](https://mlsanity.com/paper/2110.06874/) (2021)
- [Automated essay scoring using efficient transformer-based language models](https://mlsanity.com/paper/2102.13136/) (2021)
- [Improving Performance of Automated Essay Scoring by using back-translation essays and adjusted scores](https://mlsanity.com/paper/2203.00354/) (2022)

## Built on by (most cited first, 3 of 3)

- [When Automated Assessment Meets Automated Content Generation: Examining Text Quality in the Era of GPTs](https://mlsanity.com/paper/2309.14488/) (2023)
- [Phrase-Level Adversarial Training for Mitigating Bias in Neural Network-based Automatic Essay Scoring](https://mlsanity.com/paper/2409.04795/) (2024)
- [Calibrating Generative AI to Produce Realistic Essays for Data Augmentation](https://mlsanity.com/paper/2602.06772/) (2026)

## Abstract (arXiv)

Automated essay scoring is one of the most important problem in Natural Language Processing. It has been explored for a number of years, and it remains partially solved. In addition to its economic and educational usefulness, it presents research problems. Transfer learning has proved to be beneficial in NLP. Data augmentation techniques have also helped build state-of-the-art models for automated essay scoring. Many works in the past have attempted to solve this problem by using RNNs, LSTMs, etc. This work examines the transformer models like BERT, RoBERTa, etc. We empirically demonstrate the effectiveness of transformer models and data augmentation for automated essay grading across many topics using a single model.