# Specialists or Generalists? Multi-Agent and Single-Agent LLMs for Essay Grading

arXiv: https://arxiv.org/abs/2601.22386 · HTML: https://mlsanity.com/paper/2601.22386/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 11 of its 18 references that are in the map.
- First posted on arXiv 29 Jan 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 11 of 11)

- [What Did I Do Wrong? Quantifying LLMs' Sensitivity and Consistency to Prompt Engineering](https://mlsanity.com/paper/2406.12334/) (2024)
- [Fine-tuning ChatGPT for Automatic Scoring](https://mlsanity.com/paper/2310.10072/) (2023)
- [Can Large Language Models Automatically Score Proficiency of Written Essays?](https://mlsanity.com/paper/2403.06149/) (2024)
- [Rationale Behind Essay Scores: Enhancing S-LLM's Multi-Trait Essay Scoring with Rationale Generated by LLMs](https://mlsanity.com/paper/2410.14202/) (2024)
- [Can AI grade your essays? A comparative analysis of large language models and teacher ratings in multidimensional essay scoring](https://mlsanity.com/paper/2411.16337/) (2024)
- [Grade Like a Human: Rethinking Automated Assessment with Large Language Models](https://mlsanity.com/paper/2405.19694/) (2024)
- [AutoSCORE: Enhancing Automated Scoring with Multi-Agent Large Language Models via Structured Component Recognition](https://mlsanity.com/paper/2509.21910/) (2025)
- [CAFES: A Collaborative Multi-Agent Framework for Multi-Granular Multimodal Essay Scoring](https://mlsanity.com/paper/2505.13965/) (2025)
- [Single-Agent vs. Multi-Agent LLM Strategies for Automated Student Reflection Assessment](https://mlsanity.com/paper/2504.05716/) (2025)
- ["My Grade is Wrong!": A Contestable AI Framework for Interactive Feedback in Evaluating Student Essays](https://mlsanity.com/paper/2409.07453/) (2024)
- [Artificial Intelligence Bias on English Language Learners in Automatic Scoring](https://mlsanity.com/paper/2505.10643/) (2025)

## Abstract (arXiv)

Automated essay scoring (AES) systems increasingly rely on large language models, yet little is known about how architectural choices shape their performance across different essay quality levels. This paper evaluates single-agent and multi-agent LLM architectures for essay grading using the ASAP 2.0 corpus. Our multi-agent system decomposes grading into three specialist agents (Content, Structure, Language) coordinated by a Chairman Agent that implements rubric-aligned logic including veto rules and score capping. We test both architectures in zero-shot and few-shot conditions using GPT-5.1. Results show that the multi-agent system is significantly better at identifying weak essays while the single-agent system performs better on mid-range essays. Both architectures struggle with high-quality essays. Critically, few-shot calibration emerges as the dominant factor in system performance -- providing just two examples per score level improves QWK by approximately 26% for both architectures. These findings suggest architectural choice should align with specific deployment priorities, with multi-agent AI particularly suited for diagnostic screening of at-risk students, while single-agent models provide a cost-effective solution for general assessment.