mlsanityData to 7 Oct 2026

Generative Language Models with Retrieval Augmented Generation for Automated Short Answer Scoring

arXiv 2408.03811 ↗First posted 7 Aug 2024cs.CL cs.AI cs.IRcited by 0

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Automated Short Answer Scoring (ASAS) is a critical component in educational assessment. While traditional ASAS systems relied on rule-based algorithms or complex deep learning methods, recent advancements in Generative Language Models (GLMs) offer new opportunities for improvement. This study explores the application of GLMs to ASAS, leveraging their off-the-shelf capabilities and performance in various domains. We propose a novel pipeline that combines vector databases, transformer-based encoders, and GLMs to enhance short answer scoring accuracy. Our approach stores training responses in a vector database, retrieves semantically similar responses during inference, and employs a GLM to analyze these responses and determine appropriate scores. We further optimize the system through fine-tuned retrieval processes and prompt engineering. Evaluation on the SemEval 2013 dataset demonstrates a significant improvement on the SCIENTSBANK 3-way and 2-way tasks compared to existing methods, highlighting the potential of GLMs in advancing ASAS technology.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 10 of its 37 references that are in the map. First posted on arXiv 7 Aug 2024.

Builds on

10 of 37 references.

References · 10

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  7. 2017Reading Wikipedia to Answer Open-Domain Questionscited by 777
  8. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
  9. 2022Retrieval-Enhanced Machine Learningcited by 46
  10. 2024Automated Text Scoring in the Age of Generative AI for the GPU-poorcited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Beyond Scores: A Modular RAG-Based System for Automatic Short Answer Scoring with Feedback
  2. 2025SAS-Bench: A Fine-Grained Benchmark for Evaluating Short Answer Scoring with Large Language Models
  3. 2024The Future of Learning in the Age of Generative AI: Automated Question Generation and Assessment with Large Language Models
  4. 2025SteLLA: A Structured Grading System Using LLMs with RAG
  5. 2025Language Models are Few-Shot Graders
  6. 2024Automated Evaluation of Retrieval-Augmented Language Models with Task-Specific Exam Generation
  7. 2025Enhancing LLM-Based Short Answer Grading with Retrieval-Augmented Generation
  8. 2025RAG-RL: Advancing Retrieval-Augmented Generation via RL and Curriculum Learning