mlsanityData to 14 Oct 2026

MAGIC: Multi-Agent Argumentation and Grammar Integrated Critiquer

arXiv 2506.13037 ↗First posted 16 Jun 2025cs.AIcited by 2

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Automated Essay Scoring (AES) and Automatic Essay Feedback (AEF) systems aim to reduce the workload of human raters in educational assessment. However, most existing systems prioritize numerical scoring accuracy over feedback quality and are primarily evaluated on pre-secondary school level writing. This paper presents Multi-Agent Argumentation and Grammar Integrated Critiquer (MAGIC), a framework using five specialized agents to evaluate prompt adherence, persuasiveness, organization, vocabulary, and grammar for both holistic scoring and detailed feedback generation. To support evaluation at the college level, we collated a dataset of Graduate Record Examination (GRE) practice essays with expert-evaluated scores and feedback. MAGIC achieves substantial to near-perfect scoring agreement with humans on the GRE data, outperforming baseline LLM models while providing enhanced interpretability through its multi-agent approach. We also compare MAGIC's feedback generation capabilities against ground truth human feedback and baseline models, finding that MAGIC achieves strong feedback quality and naturalness.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 12 of its 53 references that are in the map. First posted on arXiv 16 Jun 2025.

Builds on

12 of 53 references.

References · 12

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
  3. 2016Parsing Argumentation Structures in Persuasive Essayscited by 101
  4. 2024Exploring LLM Prompting Strategies for Joint Essay Scoring and Feedback Generationcited by 63
  5. 2023Bridging the Novice-Expert Gap via Models of Decision-Making: A Case Study on Remediating Math Mistakescited by 52
  6. 2024Unleashing Large Language Models' Proficiency in Zero-shot Essay Scoringcited by 33
  7. 2023LLM-as-a-tutor in EFL Writing Education: Focusing on Evaluation of Student-LLM Interactioncited by 29
  8. 2018Automatic Short Answer Grading and Feedback Using Text Mining Methodscited by 11
  9. 2024Are Large Language Models Good Essay Graders?cited by 11
  10. 2024Evaluating Language Models for Generating and Judging Programming Feedbackcited by 8
  11. 2022Automated Evaluation for Student Argumentative Writing: A Surveycited by 3
  12. 2025Evaluating LLMs and Pre-trained Models for Text Summarization Across Diverse Datasetscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025LLM Agents at the Roundtable: A Multi-Perspective and Dialectical Reasoning Framework for Essay Scoring
  2. 2025CAFES: A Collaborative Multi-Agent Framework for Multi-Granular Multimodal Essay Scoring
  3. 2026Specialists or Generalists? Multi-Agent and Single-Agent LLMs for Essay Grading
  4. 2026FeedEval: Pedagogically Aligned Evaluation of LLM-Generated Essay Feedback
  5. 2025Single-Agent vs. Multi-Agent LLM Strategies for Automated Student Reflection Assessment
  6. 2026MADRAG: Multi-Agent Debate with Retrieval-Augmented Generation for Training-Free Analytic Essay Scoring
  7. 2026The Order Matters: Sequential Fine-Tuning of LLaMA for Coherent Automated Essay Scoring
  8. 2025EssayJudge: A Multi-Granular Benchmark for Assessing Automated Essay Scoring Capabilities of Multimodal Large Language Models