# Generating AI Literacy MCQs: A Multi-Agent LLM Approach

arXiv: https://arxiv.org/abs/2412.00970 · HTML: https://mlsanity.com/paper/2412.00970/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Educational agents and math problem solving, one of 16 directions in LLM agents and tool use.
- Builds on 5 of its 10 references that are in the map.
- First posted on arXiv 1 Dec 2024.

## Placed in

- LLM agents and tool use / [Educational agents and math problem solving](https://mlsanity.com/agents/educational-agents-and-math-problem-solving/)

## Builds on (most cited first, 5 of 5)

- [Enhancing LLM-Based Feedback: Insights from Intelligent Tutoring Systems and the Learning Sciences](https://mlsanity.com/paper/2405.04645/) (2024)
- [A Comparative Study of AI-Generated (GPT-4) and Human-crafted MCQs in Programming Education](https://mlsanity.com/paper/2312.03173/) (2023)
- [Automated Educational Question Generation at Different Bloom's Skill Levels using Large Language Models: Strategies and Evaluation](https://mlsanity.com/paper/2408.04394/) (2024)
- [On the application of Large Language Models for language teaching and assessment technology](https://mlsanity.com/paper/2307.08393/) (2023)
- [Assessing the Quality of Multiple-Choice Questions Using GPT-4 and Rule-Based Methods](https://mlsanity.com/paper/2307.08161/) (2023)

## Built on by (most cited first, 6 of 6)

- [Enabling Multi-Agent Systems as Learning Designers: Applying Learning Sciences to AI Instructional Design](https://mlsanity.com/paper/2508.16659/) (2025)
- [Prompting Strategies for Language Model-Based Item Generation in K-12 Education: Bridging the Gap Between Small and Large Language Models](https://mlsanity.com/paper/2508.20217/) (2025)
- [CODE-GEN: A Human-in-the-Loop RAG-Based Agentic AI System for Multiple-Choice Question Generation](https://mlsanity.com/paper/2604.03926/) (2026)
- [A Scoping Review of Large Language Model-Based Pedagogical Agents](https://mlsanity.com/paper/2604.12253/) (2026)
- [EduAgentQG: Multi-Agent Personalized Mathematics Question Generation with Explicit Diversity and Objective-Aware Evaluation](https://mlsanity.com/paper/2511.11635/) (2025)
- [TeachMateGPT: A Multi-Agent Knowledge-Grounded Framework for Pedagogical Assessment Generation from Science Curriculum Materials](https://mlsanity.com/paper/2608.13708/) (2026)

## Abstract (arXiv)

Artificial intelligence (AI) is transforming society, making it crucial to prepare the next generation through AI literacy in K-12 education. However, scalable and reliable AI literacy materials and assessment resources are lacking. To address this gap, our study presents a novel approach to generating multiple-choice questions (MCQs) for AI literacy assessments. Our method utilizes large language models (LLMs) to automatically generate scalable, high-quality assessment questions. These questions align with user-provided learning objectives, grade levels, and Bloom's Taxonomy levels. We introduce an iterative workflow incorporating LLM-powered critique agents to ensure the generated questions meet pedagogical standards. In the preliminary evaluation, experts expressed strong interest in using the LLM-generated MCQs, indicating that this system could enrich existing AI literacy materials and provide a valuable addition to the toolkit of K-12 educators.