From Memorization to Creation: Evaluating the Cognitive Depth of LLM-Generated Educational Questions
Not on the map. Why?
Abstract (arXiv)
While LLMs show promise in automating educational content creation, their ability to generate questions that stimulate higher-order thinking remains understudied. This work evaluates six widely-used LLMs through a Bloom's Taxonomy lens, focusing on their capacity to transcend rote memorization and achieve cognitive leaps. Using a hybrid human--AI evaluation protocol, we generate and analyze 20,700 questions across computer science, K--12 math, and social-science domains. Key contributions include: (1) a fine-grained prompting strategy that reduces question repetitiveness by 24.45% for Qwen2.5-7B-Instruct, and increases the proportion of higher-order cognitive level outputs by 11.53% for InternLM3-8B-Instruct; (2) quantitative metrics for cognitive shift intensity (CogShift) and category drift, revealing InternLM3's superior performance in multi-level transitions; (3) an interpretability analysis revealing metric-level correlations that enhance the transparency of Chain-of-Thought prompting. Our findings highlight the importance of cognitive-aware prompt design and provide benchmarks for deploying LLMs in personalized learning systems.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 40 references that are in the map. First posted on arXiv 6 May 2026.
Builds on
18 of 40 references.
References · 18
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
- 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
- 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,121
- 2024InternLM2 Technical Reportcited by 397
- 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 346
- 2024Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutorscited by 41
- 2024Book2Dial: Generating Teacher-Student Interactions from Textbooks for Cost-Effective Development of Educational Chatbotscited by 25
- 2022Selecting Better Samples from Pre-trained LLMs: A Case Study on Question Generationcited by 17
- 2024How Teachers Can Use Large Language Models and Bloom's Taxonomy to Create Educational Quizzescited by 17
6 more
- 2022Grow-and-Clip: Informative-yet-Concise Evidence Distillation for Answer Explanationcited by 16
- 2022Quiz Design Task: Helping Teachers Create Quizzes with Automated Question Generationcited by 12
- 2023Prompt Space Optimizing Few-shot Reasoning Success with Large Language Modelscited by 11
- 2023Empowering Private Tutoring by Chaining Large Language Modelscited by 9
- 2024Awakening Augmented Generation: Learning to Awaken Internal Knowledge of Large Language Models for Question Answeringcited by 7
- 2022Pre-Training With Scientific Text Improves Educational Question Generationcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024The Future of Learning in the Age of Generative AI: Automated Question Generation and Assessment with Large Language Models
- 2024Exploring the Capabilities of Prompted Large Language Models in Educational and Assessment Applications
- 2024Unlocking Structured Thinking in Language Models with Cognitive Prompting
- 2024uTeBC-NLP at SemEval-2024 Task 9: Can LLMs be Lateral Thinkers?
- 2024AI-Assisted Generation of Difficult Math Questions
- 2024Leveraging Large Language Models to Generate Course-specific Semantically Annotated Learning Objects
- 2025DeepQuestion: Systematic Generation of Real-World Challenges for Evaluating LLMs Performance
- 2026Orchestrating LLM Agents for Scientific Research: A Pilot Study of Multiple Choice Question (MCQ) Generation and Evaluation