# Generating Multiple-Choice Knowledge Questions with Interpretable Difficulty Estimation using Knowledge Graphs and Large Language Models

arXiv: https://arxiv.org/abs/2604.10748 · HTML: https://mlsanity.com/paper/2604.10748/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 36 references that are in the map.
- First posted on arXiv 12 Apr 2026.

## Builds on (most cited first, 13 of 13)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Open-World Knowledge Graph Completion](https://mlsanity.com/paper/1711.03438/) (2017)
- [Guiding the Growth: Difficulty-Controllable Question Generation through Step-by-Step Rewriting](https://mlsanity.com/paper/2105.11698/) (2021)
- [Fast and Accurate Network Embeddings via Very Sparse Random Projection](https://mlsanity.com/paper/1908.11512/) (2019)
- [Toward Subgraph-Guided Knowledge Graph Question Generation with Graph Neural Networks](https://mlsanity.com/paper/2004.06015/) (2020)
- [Zero-Shot Question Generation from Knowledge Graphs for Unseen Predicates and Entity Types](https://mlsanity.com/paper/1802.06842/) (2018)
- [Rethinking Knowledge Graph Evaluation Under the Open-World Assumption](https://mlsanity.com/paper/2209.08858/) (2022)
- [Knowledge Questions from Knowledge Graphs](https://mlsanity.com/paper/1610.09935/) (2016)
- [Exploring Automated Distractor Generation for Math Multiple-choice Questions via Large Language Models](https://mlsanity.com/paper/2404.02124/) (2024)
- [Prompting Large Language Models with Knowledge Graphs for Question Answering Involving Long-tail Facts](https://mlsanity.com/paper/2405.06524/) (2024)
- [Leaf: Multiple-Choice Question Generation](https://mlsanity.com/paper/2201.09012/) (2022)
- [Reasoning on Efficient Knowledge Paths:Knowledge Graph Guides Large Language Model for Domain Question Answering](https://mlsanity.com/paper/2404.10384/) (2024)
- [Are You Doubtful? Oh, It Might Be Difficult Then! Exploring the Use of Model Uncertainty for Question Difficulty Estimation](https://mlsanity.com/paper/2412.11831/) (2024)

## Abstract (arXiv)

Generating multiple-choice questions (MCQs) with difficulty estimation remains challenging in automated MCQ-generation systems used in adaptive, AI-assisted education. This study proposes a novel methodology for generating MCQs with difficulty estimation from the input documents by utilizing knowledge graphs (KGs) and large language models (LLMs). Our approach uses an LLM to construct a KG from input documents, from which MCQs are then systematically generated. Each MCQ is generated by selecting a node from the KG as the key, sampling a related triple or quintuple -- optionally augmented with an extra triple -- and prompting an LLM to generate a corresponding stem from these graph components. Distractors are then selected from the KG. For each MCQ, nine difficulty signals are computed and combined into a unified difficulty score using a data-driven approach. Experimental results demonstrate that our method generates high-quality MCQs whose difficulty estimation is interpretable and aligns with human perceptions. Our approach improves automated MCQ generation by integrating structured knowledge representations with LLMs and a data-driven difficulty estimation model.