# Get It Scored Using AutoSAS -- An Automated System for Scoring Short Answers

arXiv: https://arxiv.org/abs/2012.11243 · HTML: https://mlsanity.com/paper/2012.11243/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Dec 2020.

## Built on by (most cited first, 18 of 18)

- [What all do audio transformer models hear? Probing Acoustic Representations for Language Delivery and its Structure](https://mlsanity.com/paper/2101.00387/) (2021)
- [Survey on Automated Short Answer Grading with Deep Learning: from Word Embeddings to Transformers](https://mlsanity.com/paper/2204.03503/) (2022)
- [Automated Text Scoring in the Age of Generative AI for the GPU-poor](https://mlsanity.com/paper/2407.01873/) (2024)
- [Speaker-Conditioned Hierarchical Modeling for Automated Speech Scoring](https://mlsanity.com/paper/2109.00928/) (2021)
- [Short-answer scoring with ensembles of pretrained language models](https://mlsanity.com/paper/2202.11558/) (2022)
- [Balancing Cost and Quality: An Exploration of Human-in-the-loop Frameworks for Automated Short Answer Scoring](https://mlsanity.com/paper/2206.08288/) (2022)
- [Automated Speech Scoring System Under The Lens: Evaluating and interpreting the linguistic cues for language proficiency](https://mlsanity.com/paper/2111.15156/) (2021)
- [AES Systems Are Both Overstable And Oversensitive: Explaining Why And Proposing Defenses](https://mlsanity.com/paper/2109.11728/) (2021)
- [Using Sampling to Estimate and Improve Performance of Automated Scoring Systems with Guarantees](https://mlsanity.com/paper/2111.08906/) (2021)
- [Span Classification with Structured Information for Disfluency Detection in Spoken Utterances](https://mlsanity.com/paper/2203.16028/) (2022)
- [Beyond Scores: A Modular RAG-Based System for Automatic Short Answer Scoring with Feedback](https://mlsanity.com/paper/2409.20042/) (2024)
- [MINIMAL: Mining Models for Data Free Universal Adversarial Triggers](https://mlsanity.com/paper/2109.12406/) (2021)
- [H-AES: Towards Automated Essay Scoring for Hindi](https://mlsanity.com/paper/2302.14635/) (2023)
- [Anna Karenina Strikes Again: Pre-Trained LLM Embeddings May Favor High-Performing Learners](https://mlsanity.com/paper/2406.06599/) (2024)
- [Reducing the Cost: Cross-Prompt Pre-Finetuning for Short Answer Scoring](https://mlsanity.com/paper/2408.13966/) (2024)
- [Enhancing Security and Strengthening Defenses in Automated Short-Answer Grading Systems](https://mlsanity.com/paper/2505.00061/) (2025)
- [Interpretability from the Ground Up: Stakeholder-Centric Design of Automated Scoring in Educational Assessments](https://mlsanity.com/paper/2511.17069/) (2025)
- [Is Our Chatbot Telling Lies? Assessing Correctness of an LLM-based Dutch Support Chatbot](https://mlsanity.com/paper/2411.00034/) (2024)

## Abstract (arXiv)

In the era of MOOCs, online exams are taken by millions of candidates, where scoring short answers is an integral part. It becomes intractable to evaluate them by human graders. Thus, a generic automated system capable of grading these responses should be designed and deployed. In this paper, we present a fast, scalable, and accurate approach towards automated Short Answer Scoring (SAS). We propose and explain the design and development of a system for SAS, namely AutoSAS. Given a question along with its graded samples, AutoSAS can learn to grade that prompt successfully. This paper further lays down the features such as lexical diversity, Word2Vec, prompt, and content overlap that plays a pivotal role in building our proposed model. We also present a methodology for indicating the factors responsible for scoring an answer. The trained model is evaluated on an extensively used public dataset, namely Automated Student Assessment Prize Short Answer Scoring (ASAP-SAS). AutoSAS shows state-of-the-art performance and achieves better results by over 8% in some of the question prompts as measured by Quadratic Weighted Kappa (QWK), showing performance comparable to humans.