MADRAG: Multi-Agent Debate with Retrieval-Augmented Generation for Training-Free Analytic Essay Scoring
On the map: Retrieval-augmented generation → Agentic RAG, agents and memory
also in 1 more
Abstract (arXiv)
We present MADRAG, a training-free framework for analytic essay scoring that combines multi-agent reasoning with retrieval-augmented grounding. Unlike standard LLM-as-judge approaches, which are prone to bias and unstable scoring, MADRAG decomposes evaluation into an interactive process: an Advocate identifies strengths, a Skeptic critiques weaknesses, and a Judge aggregates their arguments into a final score. Crucially, the Judge is augmented with rubric-aligned exemplar retrieval, enabling calibration through comparison with scored examples. Our results show that MADRAG significantly outperforms prompt-based baselines while approaching the performance of supervised systems without requiring task-specific training. Ablation studies demonstrate that retrieval drives calibration gains, while debate improves reasoning on higher-level traits. Our findings highlight the complementary roles of structured interaction and external memory in reliable LLM-based evaluation.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agentic RAG, agents and memory, one of 15 directions in Retrieval-augmented generation. Builds on 18 of its 34 references that are in the map. First posted on arXiv 4 Jun 2026.
Builds on
18 of 34 references.
References · 18
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
- 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
- 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 372
- 2022On the Use of BERT for Automated Essay Scoring: Joint Learning of Multi-Scale Essay Representationcited by 46
- 2025Evaluating Scoring Bias in LLM-as-a-Judgecited by 42
- 2024Can Large Language Models Automatically Score Proficiency of Written Essays?cited by 34
- 2021Many Hands Make Light Work: Using Essay Traits to Automatically Score Essayscited by 25
- 2023Prompt- and Trait Relation-aware Cross-prompt Essay Trait Scoringcited by 24
6 more
- 2025Enhancing Multi-Agent Debate System Performance via Confidence Expressioncited by 18
- 2024Are Large Language Models Good Essay Graders?cited by 11
- 2024Autoregressive Multi-trait Essay Scoring via Reinforcement Learning with Scoring-aware Multiple Rewardscited by 11
- 2025LCES: Zero-shot Automated Essay Scoring via Pairwise Comparisons Using Large Language Modelscited by 8
- 2025CAFES: A Collaborative Multi-Agent Framework for Multi-Granular Multimodal Essay Scoringcited by 7
- 2024Says Who? Effective Zero-Shot Annotation of Focalizationcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025LLM Agents at the Roundtable: A Multi-Perspective and Dialectical Reasoning Framework for Essay Scoring
- 2025R-Debater: Retrieval-Augmented Debate Generation through Argumentative Memory
- 2026Remember and Reweight: Enhancing Multi-Agent Debate with Experience Memory and Confidence Estimation
- 2026To Isolate or to Score? Model-Adaptive Assessment for Cost-Efficient Multi-Agent RAG
- 2025MAGIC: Multi-Agent Argumentation and Grammar Integrated Critiquer
- 2026Multi-Agent Dialectical Refinement for Enhanced Argument Classification
- 2026Specialists or Generalists? Multi-Agent and Single-Agent LLMs for Essay Grading
- 2026The Confident Liar: Diagnosing Multi-Agent Debate with Log-Probabilities and LLM-as-Judge