mlsanityData to 14 Oct 2026

When LLM Essays Outscore Student Essays: What a Korean Writing Rubric Rewards and Where Readers Disagree

arXiv 2601.19913 ↗First posted 6 Jan 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

LLMs now help students plan, draft, and revise essays. Educational assessment therefore faces a basic question: how should student and LLM writing be compared? Rubrics assign points to content, organization, and expression. Their total can still hide which criteria drive the comparison, where ratings approach the maximum, and where readers disagree. We therefore conducted a secondary, post hoc audit of a Korean writing study with source-informed scoring. Three Korean language and literature majors used a 16-criterion, 100-point rubric to score six student essays and 24 essays from four LLMs prompted for three school levels. After source disclosure, they helped develop the rubric and, according to the protocol, scored the Phase 2 essays without gold labels while recording source judgments. The pooled LLM mean exceeds the student mean by 18.35 points. Orthographic norms and genre-appropriate register contribute 8.19 points, or 44.7% of the gap. All 72 LLM ratings reach the orthographic maximum, and 70 reach the register maximum. The student source mean is highest or tied highest on both creativity criteria and natural Korean phrasing, where reader agreement is weak. Criterion-level auditing therefore offers a deeper account of human and LLM writing than the total alone.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 7 of its 22 references that are in the map. First posted on arXiv 6 Jan 2026.

Builds on

7 of 22 references.

References · 7

  1. 2021All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Textcited by 287
  2. 2022Human heuristics for AI-generated language are flawedcited by 105
  3. 2022Real or Fake Text?: Investigating Human Ability to Detect Boundaries Between Human-Written and Machine-Generated Textcited by 63
  4. 2025People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated textcited by 40
  5. 2025The Reader is the Metric: How Textual Features and Reader Profiles Explain Conflicting Evaluations of AI Creative Writingcited by 8
  6. 2025KatFishNet: Detecting LLM-Generated Korean Text through Linguistic Feature Analysiscited by 6
  7. 2025Linguistic and Embedding-Based Profiling of Texts generated by Humans and Large Language Modelscited by 6

Built on by

Not cited yet.