# From Model to Classroom: Evaluating Generated MCQs for Portuguese with Narrative and Difficulty Concerns

arXiv: https://arxiv.org/abs/2506.15598 · HTML: https://mlsanity.com/paper/2506.15598/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 69 references that are in the map.
- First posted on arXiv 18 Jun 2025.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [RACE: Large-scale ReAding Comprehension Dataset From Examinations](https://mlsanity.com/paper/1704.04683/) (2017)
- [Fantastic Questions and Where to Find Them: FairytaleQA -- An Authentic Dataset for Narrative Comprehension](https://mlsanity.com/paper/2203.13947/) (2022)
- [Guiding the Growth: Difficulty-Controllable Question Generation through Step-by-Step Rewriting](https://mlsanity.com/paper/2105.11698/) (2021)
- [Automated Educational Question Generation at Different Bloom's Skill Levels using Large Language Models: Strategies and Evaluation](https://mlsanity.com/paper/2408.04394/) (2024)
- [Educational Question Generation of Children Storybooks via Question Type Distribution Learning and Event-Centric Summarization](https://mlsanity.com/paper/2203.14187/) (2022)
- [Distractor Generation in Multiple-Choice Tasks: A Survey of Methods, Datasets, and Evaluation](https://mlsanity.com/paper/2402.01512/) (2024)
- [Generating Distractors for Reading Comprehension Questions from Real Examinations](https://mlsanity.com/paper/1809.02768/) (2018)
- [Advancing Neural Encoding of Portuguese with Transformer Albertina PT-*](https://mlsanity.com/paper/2305.06721/) (2023)
- [How Useful are Educational Questions Generated by Large Language Models?](https://mlsanity.com/paper/2304.06638/) (2023)
- [Question Generation for Reading Comprehension Assessment by Modeling How and What to Ask](https://mlsanity.com/paper/2204.02908/) (2022)
- [Designing for human-AI complementarity in K-12 education](https://mlsanity.com/paper/2104.01266/) (2021)
- [Towards Process-Oriented, Modular, and Versatile Question Generation that Meets Educational Needs](https://mlsanity.com/paper/2205.00355/) (2022)
- [Distractor generation for multiple-choice questions with predictive prompting and large language models](https://mlsanity.com/paper/2307.16338/) (2023)
- [Co-Attention Hierarchical Network: Generating Coherent Long Distractors for Reading Comprehension](https://mlsanity.com/paper/1911.08648/) (2019)
- [Improving Reading Comprehension Question Generation with Data Augmentation and Overgenerate-and-rank](https://mlsanity.com/paper/2306.08847/) (2023)
- [Simple or Complex? Complexity-Controllable Question Generation with Soft Templates and Deep Mixture of Experts Model](https://mlsanity.com/paper/2110.06560/) (2021)
- [ptt5-v2: A Closer Look at Continued Pretraining of T5 Models for the Portuguese Language](https://mlsanity.com/paper/2406.10806/) (2024)
- [Towards Diverse and Effective Question-Answer Pair Generation from Children Storybooks](https://mlsanity.com/paper/2306.06605/) (2023)
- [Towards Enriched Controllability for Educational Question Generation](https://mlsanity.com/paper/2306.14917/) (2023)
- [On Few-Shot Prompting for Controllable Question-Answer Generation in Narrative Comprehension](https://mlsanity.com/paper/2404.02800/) (2024)
- [FairytaleQA Translated: Enabling Educational Question and Answer Generation in Less-Resourced Languages](https://mlsanity.com/paper/2406.04233/) (2024)

## Abstract (arXiv)

While MCQs are valuable for learning and evaluation, manually creating them with varying difficulty levels and targeted reading skills remains a time-consuming and costly task. Recent advances in generative AI provide an opportunity to automate MCQ generation efficiently. However, assessing the actual quality and reliability of generated MCQs has received limited attention -- particularly regarding cases where generation fails. This aspect becomes particularly important when the generated MCQs are meant to be applied in real-world settings. Additionally, most MCQ generation studies focus on English, leaving other languages underexplored. This paper investigates the capabilities of current generative models in producing MCQs for reading comprehension in Portuguese, a morphologically rich language. Our study focuses on generating MCQs that align with curriculum-relevant narrative elements and span different difficulty levels. We evaluate these MCQs through expert review and by analyzing the psychometric properties extracted from student responses to assess their suitability for elementary school students. Our results show that current models can generate MCQs of comparable quality to human-authored ones. However, we identify issues related to semantic clarity and answerability. Also, challenges remain in generating distractors that engage students and meet established criteria for high-quality MCQ option design.