mlsanityData to 14 Oct 2026

TeachGrade: Training-Free Short Answer Scoring with Teacher Evidence

arXiv 2610.14900 ↗First posted 10 Oct 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Multi-subject, multi-step scoring requires consistent partial credit across grading conventions. We introduce TeachGrade, a training-free framework using teacher-graded examples. The Task Adapt module builds two frozen memories: Skill captures contextual teacher preferences beyond the rubric; CaseBank supplies local evidence. PreJudge provides retrieval cues; FinalJudge regrades the answer without preliminary scores. On 3,829 SAS-Bench answers across 12 tasks, official macro collaborative consistency score (CCS) rises from Native's 67.99 to 81.32 (+13.33 points), and pooled step exact match from 54.85% to 62.71%. Macro quadratic weighted kappa (QWK) reaches 84.71, exceeding Native, OpenJudge, ProofGrader, and Trojsten by 9.52, 3.50, 11.37, and 17.43 points, respectively, and leading on 8 tasks. Local Qwen3.8-27B gains 8.83 CCS and 4.08 QWK points, improving CCS on all 12 tasks. QWK gains over Native span 6.38--14.65 points on Portuguese automatic essay scoring (AES) and 2.16--2.24 on ASAP-SAS across label views, with lower mean absolute error. These results support reuse across languages, response formats, and backbones without weight updates.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 38 references that are in the map. First posted on arXiv 10 Oct 2026.

Builds on

21 of 38 references.

References · 21

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2023Let's Verify Step by Stepcited by 3,789
  4. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  5. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  6. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  7. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  8. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  9. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 928
  10. 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
  11. 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 575
  12. 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 446
9 more
  1. 2023JudgeLM: Fine-tuned Large Language Models are Scalable Judgescited by 367
  2. 2024JudgeBench: A Benchmark for Evaluating LLM-based Judgescited by 318
  3. 2024ProcessBench: Identifying Process Errors in Mathematical Reasoningcited by 292
  4. 2024Trust or Escalate: LLM Judges with Provable Guarantees for Human Agreementcited by 105
  5. 2024Limits to scalable evaluation at the frontier: LLM as Judge won't beat twice the datacited by 51
  6. 2025Auto-Rubric: Learning From Implicit Weights to Explicit Rubrics for Reward Modelingcited by 30
  7. 2025Does Context Matter? ContextualJudgeBench for Evaluating LLM-based Judges in Contextual Settingscited by 24
  8. 2025Reliable Fine-Grained Evaluation of Natural Language Math Proofscited by 18
  9. 2024Generative Language Models with Retrieval Augmented Generation for Automated Short Answer Scoringcited by 1

Built on by

Not cited yet.