mlsanityData to 14 Oct 2026

Measuring the Partial-Credit Gap: A Strict Benchmark on Vietnam's 2025 Convex Marking Scheme

arXiv 2608.18336 ↗First posted 18 Aug 2026cs.AI cs.CYcited by 0

Not on the map. Why?

Abstract (arXiv)

When evaluating language models on human exams, benchmarks typically score each response as right or wrong and report the overall accuracy. This approach assumes that partial knowledge is worth proportional credit, an assumption that fails when an examination uses a non-additive grading scheme. The 2025 reform of Vietnam's National High School Graduation Examination demonstrates the cost of this substitution. In Part II of the exam, candidates evaluate four true/false statements per question. The grading is convex: the number of correct statements earns 0, 0.10, 0.25, 0.50, or 1.00 points. Identifying three statements correctly pays 0.50 points, not the 0.75 points that standard accuracy metrics would award. Because Part II accounts for 4.00 of the exam's 10.00 points, reporting accuracy inflates the score by rewarding partial knowledge that the state explicitly penalizes. We introduce THPT-Ladder, a benchmark of 632 items from 21 official exams across 11 subjects, graded exactly as the ministry grades its students. The ministry publishes the marks of over a million candidates, allowing us to place models directly into the human cohort. Across eight models, the official rubric pays 0.020 to 0.159 points less per Part II question than proportional credit. This shortfall changes a model's apparent competence. For Qwen3.5-27B on the 2025 History exam, a 0.042-point shortfall drops its standing from the 90th to the 77th percentile among 481,293 candidates. A model's accuracy does not predict this penalty. At Claude Sonnet 5's accuracy level, different distributions of errors yield scores varying from 0.869 to 0.932 points per question. Official marks depend on how correct statements are grouped, meaning standard benchmarks report a competence the institution would not certify.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 14 references that are in the map. First posted on arXiv 18 Aug 2026.

Builds on

12 of 14 references.

References · 12

  1. 2024ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabiccited by 88
  2. 2024Can multiple-choice questions really be useful in detecting the abilities of LLMs?cited by 69
  3. 2023Large Language Models Only Pass Primary School Exams in Indonesia: A Comprehensive Test on IndoMMLUcited by 53
  4. 2024Psychometric Alignment: Capturing Human Knowledge Distributions via Language Modelscited by 18
  5. 2023VNHSGE: VietNamese High School Graduation Examination Dataset for Large Language Modelscited by 15
  6. 2025CMPhysBench: A Benchmark for Evaluating Large Language Models in Condensed Matter Physicscited by 9
  7. 2025SinhalaMMLU: A Comprehensive Benchmark for Evaluating Multitask Language Understanding in Sinhalacited by 6
  8. 2026RadSEM: A Finding-by-Finding Metric for Clinical Consistency in Radiology Reportscited by 3
  9. 2025LaoBench: A Large-Scale Multidimensional Lao Benchmark for Large Language Modelscited by 2
  10. 2026PsyScore: A Psychometrically-Aware Framework for Trait-Adaptive Essay Scoring and ZPD-Scaffolded Feedbackcited by 2
  11. 2026SteuerLLM: Local specialized large language model for German tax law analysiscited by 1
  12. 2026Assessing the Pedagogical Readiness of Large Language Models as AI Tutors in Low-Resource Contexts: A Case Study of Nepal's K-10 Curriculumcited by 1

Built on by

Not cited yet.