# Cognitively Diverse Multiple-Choice Question Generation: A Hybrid Multi-Agent Framework with Large Language Models

arXiv: https://arxiv.org/abs/2602.03704 · HTML: https://mlsanity.com/paper/2602.03704/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Educational agents and math problem solving, one of 16 directions in LLM agents and tool use.
- Builds on 18 of its 65 references that are in the map.
- First posted on arXiv 3 Feb 2026.

## Placed in

- LLM agents and tool use / [Educational agents and math problem solving](https://mlsanity.com/agents/educational-agents-and-math-problem-solving/)

## Builds on (most cited first, 18 of 18)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Large Language Models Can Self-Improve](https://mlsanity.com/paper/2210.11610/) (2022)
- [Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm](https://mlsanity.com/paper/2102.07350/) (2021)
- [Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration](https://mlsanity.com/paper/2307.05300/) (2023)
- [Can large language models provide useful feedback on research papers? A large-scale empirical analysis](https://mlsanity.com/paper/2310.01783/) (2023)
- [Large Language Models for Education: A Survey and Outlook](https://mlsanity.com/paper/2403.18105/) (2024)
- [Applying Large Language Models and Chain-of-Thought for Automatic Scoring](https://mlsanity.com/paper/2312.03748/) (2023)
- [OctoTools: An Agentic Framework with Extensible Tools for Complex Reasoning](https://mlsanity.com/paper/2502.11271/) (2025)
- [Automated Educational Question Generation at Different Bloom's Skill Levels using Large Language Models: Strategies and Evaluation](https://mlsanity.com/paper/2408.04394/) (2024)
- [Multiple-Choice Question Generation: Towards an Automated Assessment Framework](https://mlsanity.com/paper/2209.11830/) (2022)
- [Scalable Educational Question Generation with Pre-trained Language Models](https://mlsanity.com/paper/2305.07871/) (2023)
- [Distractor generation for multiple-choice questions with predictive prompting and large language models](https://mlsanity.com/paper/2307.16338/) (2023)
- [Enhancing Distractor Generation for Multiple-Choice Questions with Retrieval Augmented Pretraining and Knowledge Graph Integration](https://mlsanity.com/paper/2406.13578/) (2024)
- [Math Multiple Choice Question Generation via Human-Large Language Model Collaboration](https://mlsanity.com/paper/2405.00864/) (2024)
- [Automated question generation and question answering from Turkish texts](https://mlsanity.com/paper/2111.06476/) (2021)
- [Automating question generation from educational text](https://mlsanity.com/paper/2309.15004/) (2023)

## Built on by (most cited first, 2 of 2)

- [Self-hosted Lecture-to-Quiz: Local LLM MCQ Generation with Deterministic Quality Control](https://mlsanity.com/paper/2603.08729/) (2026)
- [Orchestrating LLM Agents for Scientific Research: A Pilot Study of Multiple Choice Question (MCQ) Generation and Evaluation](https://mlsanity.com/paper/2602.18891/) (2026)

## Abstract (arXiv)

Recent advances in large language models (LLMs) have made automated multiple-choice question (MCQ) generation increasingly feasible; however, reliably producing items that satisfy controlled cognitive demands remains a challenge. To address this gap, we introduce ReQUESTA, a hybrid, multi-agent framework for generating cognitively diverse MCQs that systematically target text-based, inferential, and main idea comprehension. ReQUESTA decomposes MCQ authoring into specialized subtasks and coordinates LLM-powered agents with rule-based components to support planning, controlled generation, iterative evaluation, and post-processing. We evaluated the framework in a large-scale reading comprehension study using academic expository texts, comparing ReQUESTA-generated MCQs with those produced by a single-pass GPT-5 zero-shot baseline. Psychometric analyses of learner responses assessed item difficulty and discrimination, while expert raters evaluated question quality across multiple dimensions, including topic relevance and distractor quality. Results showed that ReQUESTA-generated items were consistently more challenging, more discriminative, and more strongly aligned with overall reading comprehension performance. Expert evaluations further indicated stronger alignment with central concepts and superior distractor linguistic consistency and semantic plausibility, particularly for inferential questions. These findings demonstrate that hybrid, agentic orchestration can systematically improve the reliability and controllability of LLM-based generation, highlighting workflow design as a key lever for structured artifact generation beyond single-pass prompting.