# Exploring Question-Specific Rewards for Generating Deep Questions

arXiv: https://arxiv.org/abs/2011.01102 · HTML: https://mlsanity.com/paper/2011.01102/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Nov 2020.

## Built on by (most cited first, 9 of 9)

- [Educational Question Generation of Children Storybooks via Question Type Distribution Learning and Event-Centric Summarization](https://mlsanity.com/paper/2203.14187/) (2022)
- [Improving Unsupervised Question Answering via Summarization-Informed Question Generation](https://mlsanity.com/paper/2109.07954/) (2021)
- [MCQG-SRefine: Multiple Choice Question Generation and Evaluation with Iterative Self-Critique, Correction, and Comparison Feedback](https://mlsanity.com/paper/2410.13191/) (2024)
- [QAScore -- An Unsupervised Unreferenced Metric for the Question Generation Evaluation](https://mlsanity.com/paper/2210.04320/) (2022)
- [Harnessing the Power of Prompt-based Techniques for Generating School-Level Questions using Large Language Models](https://mlsanity.com/paper/2312.01032/) (2023)
- [Explainable Multi-hop Question Generation: An End-to-End Approach without Intermediate Question Labeling](https://mlsanity.com/paper/2404.00571/) (2024)
- [Automatic Answerability Evaluation for Question Generation](https://mlsanity.com/paper/2309.12546/) (2023)
- [QA4QG: Using Question Answering to Constrain Multi-Hop Question Generation](https://mlsanity.com/paper/2202.06538/) (2022)
- [How Significant Are the Real Performance Gains? An Unbiased Evaluation Framework for GraphRAG](https://mlsanity.com/paper/2506.06331/) (2025)

## Abstract (arXiv)

Recent question generation (QG) approaches often utilize the sequence-to-sequence framework (Seq2Seq) to optimize the log-likelihood of ground-truth questions using teacher forcing. However, this training objective is inconsistent with actual question quality, which is often reflected by certain global properties such as whether the question can be answered by the document. As such, we directly optimize for QG-specific objectives via reinforcement learning to improve question quality. We design three different rewards that target to improve the fluency, relevance, and answerability of generated questions. We conduct both automatic and human evaluations in addition to a thorough analysis to explore the effect of each QG-specific reward. We find that optimizing question-specific rewards generally leads to better performance in automatic evaluation metrics. However, only the rewards that correlate well with human judgement (e.g., relevance) lead to real improvement in question quality. Optimizing for the others, especially answerability, introduces incorrect bias to the model, resulting in poor question quality. Our code is publicly available at https://github.com/YuxiXie/RL-for-Question-Generation.