# Ranking Clarifying Questions Based on Predicted User Engagement

arXiv: https://arxiv.org/abs/2103.06192 · HTML: https://mlsanity.com/paper/2103.06192/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Mar 2021.

## Built on by (most cited first, 6 of 6)

- [Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questions](https://mlsanity.com/paper/2109.05794/) (2021)
- [Evaluating Mixed-initiative Conversational Search Systems via User Simulation](https://mlsanity.com/paper/2204.08046/) (2022)
- [A Survey of Conversational Search](https://mlsanity.com/paper/2410.15576/) (2024)
- [Analysing Mixed Initiatives and Search Strategies during Conversational Search](https://mlsanity.com/paper/2109.05955/) (2021)
- [Keyword Extraction for Improved Document Retrieval in Conversational Search](https://mlsanity.com/paper/2109.05979/) (2021)
- [Leveraging Interesting Facts to Enhance User Engagement with Conversational Interfaces](https://mlsanity.com/paper/2404.06659/) (2024)

## Abstract (arXiv)

To improve online search results, clarification questions can be used to elucidate the information need of the user. This research aims to predict the user engagement with the clarification pane as an indicator of relevance based on the lexical information: query, question, and answers. Subsequently, the predicted user engagement can be used as a feature to rank the clarification panes. Regression and classification are applied for predicting user engagement and compared to naive heuristic baselines (e.g. mean) on the new MIMICS dataset [20]. An ablation study is carried out using a RankNet model to determine whether the predicted user engagement improves clarification pane ranking performance. The prediction models were able to improve significantly upon the naive baselines, and the predicted user engagement feature significantly improved the RankNet results in terms of NDCG and MRR. This research demonstrates the potential for ranking clarification panes based on lexical information only and can serve as a first neural baseline for future research to improve on. The code is available online.