# Ratas framework: A comprehensive genai-based approach to rubric-based marking of real-world textual exams

arXiv: https://arxiv.org/abs/2505.23818 · HTML: https://mlsanity.com/paper/2505.23818/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 48 references that are in the map.
- First posted on arXiv 27 May 2025.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [The Falcon Series of Open Language Models](https://mlsanity.com/paper/2311.16867/) (2023)
- [Reframing Instructional Prompts to GPTk's Language](https://mlsanity.com/paper/2109.07830/) (2021)
- [Explaining black box text modules in natural language with language models](https://mlsanity.com/paper/2305.09863/) (2023)
- [Fine-tuning ChatGPT for Automatic Scoring](https://mlsanity.com/paper/2310.10072/) (2023)
- [Prompt Design and Engineering: Introduction and Advanced Methods](https://mlsanity.com/paper/2401.14423/) (2024)
- [Putting GPT-4o to the Sword: A Comprehensive Evaluation of Language, Vision, Speech, and Multimodal Proficiency](https://mlsanity.com/paper/2407.09519/) (2024)
- [HiFT: A Hierarchical Full Parameter Fine-Tuning Strategy](https://mlsanity.com/paper/2401.15207/) (2024)
- [Automated Long Answer Grading with RiceChem Dataset](https://mlsanity.com/paper/2404.14316/) (2024)
- [Automatic assessment of text-based responses in post-secondary education: A systematic review](https://mlsanity.com/paper/2308.16151/) (2023)
- [Explainable Automatic Grading with Neural Additive Models](https://mlsanity.com/paper/2405.00489/) (2024)

## Built on by (most cited first, 1 of 1)

- [LLM-Driven Rubric-Based Assessment of Algebraic Competence in Multi-Stage Block Coding Tasks with Design and Field Evaluation](https://mlsanity.com/paper/2510.06253/) (2025)

## Abstract (arXiv)

Automated answer grading is a critical challenge in educational technology, with the potential to streamline assessment processes, ensure grading consistency, and provide timely feedback to students. However, existing approaches are often constrained to specific exam formats, lack interpretability in score assignment, and struggle with real-world applicability across diverse subjects and assessment types. To address these limitations, we introduce RATAS (Rubric Automated Tree-based Answer Scoring), a novel framework that leverages state-of-the-art generative AI models for rubric-based grading of textual responses. RATAS is designed to support a wide range of grading rubrics, enable subject-agnostic evaluation, and generate structured, explainable rationales for assigned scores. We formalize the automatic grading task through a mathematical framework tailored to rubric-based assessment and present an architecture capable of handling complex, real-world exam structures. To rigorously evaluate our approach, we construct a unique, contextualized dataset derived from real-world project-based courses, encompassing diverse response formats and varying levels of complexity. Empirical results demonstrate that RATAS achieves high reliability and accuracy in automated grading while providing interpretable feedback that enhances transparency for both students and nstructors.