mlsanityData to 14 Oct 2026

AI-Enabled Quality Assurance for Multiple-Choice Assessment Items

arXiv 2610.04267 ↗First posted 3 Oct 2026cs.CL cs.AI cs.CYcited by 0

Not on the map. Why?

Abstract (arXiv)

Generating multiple-choice questions is increasingly scalable, but establishing their assessment quality remains difficult. We present a focused narrative review of automated item-writing flaw detection, revision, psychometric screening, and NLP benchmark auditing. Database searches, citation retrieval, and nominated sources yield fourteen research reports reviewed in full text. We distinguish surface checks from content-sensitive judgments and map a 19-criterion rubric to detection methods and reported evidence. High label-level accuracy often coexists with weak positive case detection, while rubric definitions and reference standards vary. Revision evidence is mixed, and the associations reported in prior work do not establish the effects of repair. We propose evaluating quality assurance as a sequence of independently validated decisions, with criterion-specific reporting, calibrated human review, and outcome-based assessment of revisions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 5 of its 23 references that are in the map. First posted on arXiv 3 Oct 2026.

Builds on

5 of 23 references.

References · 5

  1. 2025Which of These Best Describes Multiple Choice Evaluation with LLMs? A) Forced B) Flawed C) Fixable D) All of the Abovecited by 52
  2. 2023A Comparative Study of AI-Generated (GPT-4) and Human-crafted MCQs in Programming Educationcited by 35
  3. 2023Assessing the Quality of Multiple-Choice Questions Using GPT-4 and Rule-Based Methodscited by 24
  4. 2024An Automatic Question Usability Evaluation Toolkitcited by 10
  5. 2026BenchMarker: An Education-Inspired Toolkit for Highlighting Flaws in Multiple-Choice Benchmarkscited by 5

Built on by

Not cited yet.