# Survey on Automated Short Answer Grading with Deep Learning: from Word Embeddings to Transformers

arXiv: https://arxiv.org/abs/2204.03503 · HTML: https://mlsanity.com/paper/2204.03503/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 77 references that are in the map.
- First posted on arXiv 11 Mar 2022.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [Recent Trends in Deep Learning Based Natural Language Processing](https://mlsanity.com/paper/1708.02709/) (2017)
- [Long Short-Term Memory-Networks for Machine Reading](https://mlsanity.com/paper/1601.06733/) (2016)
- [Semantics-aware BERT for Language Understanding](https://mlsanity.com/paper/1909.02209/) (2019)
- [Training Complex Models with Multi-Task Weak Supervision](https://mlsanity.com/paper/1810.02840/) (2018)
- [Attention in Natural Language Processing](https://mlsanity.com/paper/1902.02181/) (2019)
- [CAT-Gen: Improving Robustness in NLP Models via Controlled Adversarial Text Generation](https://mlsanity.com/paper/2010.02338/) (2020)
- [Improving Multi-Task Deep Neural Networks via Knowledge Distillation for Natural Language Understanding](https://mlsanity.com/paper/1904.09482/) (2019)
- [Language Model is All You Need: Natural Language Understanding as Question Answering](https://mlsanity.com/paper/2011.03023/) (2020)
- [Get It Scored Using AutoSAS -- An Automated System for Scoring Short Answers](https://mlsanity.com/paper/2012.11243/) (2020)
- [Automatic Short Answer Grading and Feedback Using Text Mining Methods](https://mlsanity.com/paper/1807.10543/) (2018)
- [Automatic Short Answer Grading via Multiway Attention Networks](https://mlsanity.com/paper/1909.10166/) (2019)
- [Evaluating Text GANs as Language Models](https://mlsanity.com/paper/1810.12686/) (2018)
- [Comparative Evaluation of Pretrained Transfer Learning Models on Automatic Short Answer Grading](https://mlsanity.com/paper/2009.01303/) (2020)
- [Joint Multi-Domain Learning for Automatic Short Answer Grading](https://mlsanity.com/paper/1902.09183/) (2019)
- [An Iterative Transfer Learning Based Ensemble Technique for Automatic Short Answer Grading](https://mlsanity.com/paper/1609.04909/) (2016)

## Built on by (most cited first, 20 of 20)

- [Applying Large Language Models and Chain-of-Thought for Automatic Scoring](https://mlsanity.com/paper/2312.03748/) (2023)
- [Towards LLM-based Autograding for Short Textual Answers](https://mlsanity.com/paper/2309.11508/) (2023)
- [Performance of the Pre-Trained Large Language Model GPT-4 on Automated Short Answer Grading](https://mlsanity.com/paper/2309.09338/) (2023)
- [Short Answer Grading Using One-shot Prompting and Text Similarity Scoring Model](https://mlsanity.com/paper/2305.18638/) (2023)
- [Disentangling Learning from Judgment: Representation Learning for Open Response Analytics](https://mlsanity.com/paper/2512.23941/) (2025)
- [Can LLMs Grade Short-Answer Reading Comprehension Questions : An Empirical Study with a Novel Dataset](https://mlsanity.com/paper/2310.18373/) (2023)
- [Autograding Mathematical Induction Proofs with Natural Language Processing](https://mlsanity.com/paper/2406.10268/) (2024)
- [Who's the Best Detective? LLMs vs. MLs in Detecting Incoherent Fourth Grade Math Answers](https://mlsanity.com/paper/2304.11257/) (2023)
- [Explainable Automatic Grading with Neural Additive Models](https://mlsanity.com/paper/2405.00489/) (2024)
- [The Rise of Artificial Intelligence in Educational Measurement: Opportunities and Ethical Challenges](https://mlsanity.com/paper/2406.18900/) (2024)
- [Confusion-Aware Rubric Optimization for LLM-based Automated Grading](https://mlsanity.com/paper/2603.00451/) (2026)
- [Reducing the Cost: Cross-Prompt Pre-Finetuning for Short Answer Scoring](https://mlsanity.com/paper/2408.13966/) (2024)
- [Using language models in the implicit automated assessment of mathematical short answer items](https://mlsanity.com/paper/2308.11006/) (2023)
- [Auditing an Automatic Grading Model with deep Reinforcement Learning](https://mlsanity.com/paper/2405.07087/) (2024)
- [Learning to Love Edge Cases in Formative Math Assessment: Using the AMMORE Dataset and Chain-of-Thought Prompting to Improve Grading Accuracy](https://mlsanity.com/paper/2409.17904/) (2024)
- [Grading Handwritten Engineering Exams with Multimodal Large Language Models](https://mlsanity.com/paper/2601.00730/) (2026)
- [S-GRADES -- Studying Generalization of Student Response Assessments in Diverse Evaluative Settings](https://mlsanity.com/paper/2603.10233/) (2026)
- [Confidence Estimation in Automatic Short Answer Grading with LLMs](https://mlsanity.com/paper/2605.00200/) (2026)
- [Estimating LLM Grading Ability and Response Difficulty in Automatic Short Answer Grading via Item Response Theory](https://mlsanity.com/paper/2605.00238/) (2026)
- [Quality-Conditioned Agreement in Automated Short Answer Scoring: Mid-Range Degradation and the Impact of Task-Specific Adaptation](https://mlsanity.com/paper/2605.07647/) (2026)

## Abstract (arXiv)

Automated short answer grading (ASAG) has gained attention in education as a means to scale educational tasks to the growing number of students. Recent progress in Natural Language Processing and Machine Learning has largely influenced the field of ASAG, of which we survey the recent research advancements. We complement previous surveys by providing a comprehensive analysis of recently published methods that deploy deep learning approaches. In particular, we focus our analysis on the transition from hand engineered features to representation learning approaches, which learn representative features for the task at hand automatically from large corpora of data. We structure our analysis of deep learning methods along three categories: word embeddings, sequential models, and attention-based methods. Deep learning impacted ASAG differently than other fields of NLP, as we noticed that the learned representations alone do not contribute to achieve the best results, but they rather show to work in a complementary way with hand-engineered features. The best performance are indeed achieved by methods that combine the carefully hand-engineered features with the power of the semantic descriptions provided by the latest models, like transformers architectures. We identify challenges and provide an outlook on research direction that can be addressed in the future