STRIVE: A Think & Improve Approach with Iterative Refinement for Enhancing Question Quality Estimation
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
Automatically assessing question quality is crucial for educators as it saves time, ensures consistency, and provides immediate feedback for refining teaching materials. We propose a novel methodology called STRIVE (Structured Thinking and Refinement with multiLLMs for Improving Verified Question Estimation) using a series of Large Language Models (LLMs) for automatic question evaluation. This approach aims to improve the accuracy and depth of question quality assessment, ultimately supporting diverse learners and enhancing educational practices. The method estimates question quality in an automated manner by generating multiple evaluations based on the strengths and weaknesses of the provided question and then choosing the best solution generated by the LLM. Then the process is improved by iterative review and response with another LLM until the evaluation metric values converge. This sophisticated method of evaluating question quality improves the estimation of question quality by automating the task of question quality evaluation. Correlation scores show that using this proposed method helps to improve correlation with human judgments compared to the baseline method. Error analysis shows that metrics like relevance and appropriateness improve significantly relative to human judgments by using STRIVE.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 6 of its 30 references that are in the map. First posted on arXiv 8 Apr 2025.
Builds on
6 of 30 references.
References · 6
- 2017Crowdsourcing Multiple Choice Science Questionscited by 742
- 2019A Hybrid Retrieval-Generation Neural Conversation Modelcited by 18
- 2021Results and Insights from Diagnostic Questions: The NeurIPS 2020 Education Challengecited by 17
- 2022QAScore -- An Unsupervised Unreferenced Metric for the Question Generation Evaluationcited by 9
- 2023Harnessing the Power of Prompt-based Techniques for Generating School-Level Questions using Large Language Modelscited by 9
- 2022Entity-Conditioned Question Generation for Robust Attention Distribution in Neural Information Retrievalcited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Generate-Then-Validate: A Novel Question Generation Approach Using Small Language Models
- 2024The Future of Learning in the Age of Generative AI: Automated Question Generation and Assessment with Large Language Models
- 2025Benchmarking Critical Questions Generation: A Challenging Reasoning Task for Large Language Models
- 2024Leveraging Large Language Models to Generate Course-specific Semantically Annotated Learning Objects
- 2024Automated Evaluation of Retrieval-Augmented Language Models with Task-Specific Exam Generation
- 2025Leveraging In-Context Learning and Retrieval-Augmented Generation for Automatic Question Generation in Educational Domains
- 2025Analysis of instruction-based LLMs' capabilities to score and judge text-input problems in an academic setting
- 2025Socratic Students: Teaching Language Models to Learn by Asking Questions