# Comparison of Large Language Models for Generating Contextually Relevant Questions

arXiv: https://arxiv.org/abs/2407.20578 · HTML: https://mlsanity.com/paper/2407.20578/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 15 references that are in the map.
- First posted on arXiv 30 Jul 2024.

## Builds on (most cited first, 5 of 5)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [How Useful are Educational Questions Generated by Large Language Models?](https://mlsanity.com/paper/2304.06638/) (2023)
- [Improving Question Generation with Multi-level Content Planning](https://mlsanity.com/paper/2310.13512/) (2023)
- [Generative AI: Implications and Applications for Education](https://mlsanity.com/paper/2305.07605/) (2023)

## Built on by (most cited first, 2 of 2)

- [Multi-Agent Retrieval-Augmented Framework for Evidence-Based Counterspeech Against Health Misinformation](https://mlsanity.com/paper/2507.07307/) (2025)
- [ConQuer: A Framework for Concept-Based Quiz Generation](https://mlsanity.com/paper/2503.14662/) (2025)

## Abstract (arXiv)

This study explores the effectiveness of Large Language Models (LLMs) for Automatic Question Generation in educational settings. Three LLMs are compared in their ability to create questions from university slide text without fine-tuning. Questions were obtained in a two-step pipeline: first, answer phrases were extracted from slides using Llama 2-Chat 13B; then, the three models generated questions for each answer. To analyze whether the questions would be suitable in educational applications for students, a survey was conducted with 46 students who evaluated a total of 246 questions across five metrics: clarity, relevance, difficulty, slide relation, and question-answer alignment. Results indicate that GPT-3.5 and Llama 2-Chat 13B outperform Flan T5 XXL by a small margin, particularly in terms of clarity and question-answer alignment. GPT-3.5 especially excels at tailoring questions to match the input answers. The contribution of this research is the analysis of the capacity of LLMs for Automatic Question Generation in education.