# An Automated Explainable Educational Assessment System Built on LLMs

arXiv: https://arxiv.org/abs/2412.13381 · HTML: https://mlsanity.com/paper/2412.13381/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 21 references that are in the map.
- First posted on arXiv 17 Dec 2024.

## Builds on (most cited first, 12 of 12)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [e-SNLI: Natural Language Inference with Natural Language Explanations](https://mlsanity.com/paper/1812.01193/) (2018)
- [Few-Shot Self-Rationalization with Natural Language Prompts](https://mlsanity.com/paper/2111.08284/) (2021)
- [The Mystery of In-Context Learning: A Comprehensive Survey on Interpretation and Analysis](https://mlsanity.com/paper/2311.00237/) (2023)
- [Eliminating Biased Length Reliance of Direct Preference Optimization via Down-Sampled KL Divergence](https://mlsanity.com/paper/2406.10957/) (2024)
- [Rationalization for Explainable NLP: A Survey](https://mlsanity.com/paper/2301.08912/) (2023)
- [Distilling ChatGPT for Explainable Automated Student Answer Assessment](https://mlsanity.com/paper/2305.12962/) (2023)
- [Calibrating LLMs with Preference Optimization on Thought Trees for Generating Rationale in Science Question Scoring](https://mlsanity.com/paper/2406.19949/) (2024)
- [AERA Chat: An Interactive Platform for Automated Explainable Student Answer Assessment](https://mlsanity.com/paper/2410.09507/) (2024)

## Built on by (most cited first, 5 of 5)

- [Beyond Static Responses: Multi-Agent LLM Systems as a New Paradigm for Social Science Research](https://mlsanity.com/paper/2506.01839/) (2025)
- [Two Heads Are Better Than One: Dual-Model Verbal Reflection at Inference-Time](https://mlsanity.com/paper/2502.19230/) (2025)
- [Sparse Activation Editing for Reliable Instruction Following in Narratives](https://mlsanity.com/paper/2505.16505/) (2025)
- [Interpretability from the Ground Up: Stakeholder-Centric Design of Automated Scoring in Educational Assessments](https://mlsanity.com/paper/2511.17069/) (2025)
- [CoLearn: An Agentic Tutor that Learns its Learner in a Human--AI Co-Learning Loop](https://mlsanity.com/paper/2609.21154/) (2026)

## Abstract (arXiv)

In this demo, we present AERA Chat, an automated and explainable educational assessment system designed for interactive and visual evaluations of student responses. This system leverages large language models (LLMs) to generate automated marking and rationale explanations, addressing the challenge of limited explainability in automated educational assessment and the high costs associated with annotation. Our system allows users to input questions and student answers, providing educators and researchers with insights into assessment accuracy and the quality of LLM-assessed rationales. Additionally, it offers advanced visualization and robust evaluation tools, enhancing the usability for educational assessment and facilitating efficient rationale verification. Our demo video can be found at https://youtu.be/qUSjz-sxlBc.