SkillQG: Learning to Generate Question for Reading Comprehension Assessment

We present textbftextttSkillQG: a question generation framework with controllable comprehension types for assessing and improving machine reading comprehension models. Existing question generation systems widely differentiate questions by textitliteral information such as question words and answer types to generate semantically relevant questions for a given context. However, they rarely consider the textitcomprehension nature of questions, i.e. the different comprehension capabilities embodied by different questions. In comparison, our textttSkillQG is able to tailor a fine-grained assessment and improvement to the capabilities of question answering models built on it. Specifically, we first frame the comprehension type of questions based on a hierarchical skill-based schema, then formulate textttSkillQG as a skill-conditioned question generator. Furthermore, to improve the controllability of generation, we augment the input text with question focus and skill-specific knowledge, which are constructed by iteratively prompting the pre-trained language models. Empirical results demonstrate that textttSkillQG outperforms baselines in terms of quality, relevance, and skill-controllability while showing a promising performance boost in downstream question answering task.