# MIMICS-Duo: Offline & Online Evaluation of Search Clarification

arXiv: https://arxiv.org/abs/2206.04417 · HTML: https://mlsanity.com/paper/2206.04417/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jun 2022.

## Built on by (most cited first, 7 of 7)

- [A Survey of Conversational Search](https://mlsanity.com/paper/2410.15576/) (2024)
- [INTERS: Unlocking the Power of Large Language Models in Search with Instruction Tuning](https://mlsanity.com/paper/2401.06532/) (2024)
- [Asking Multimodal Clarifying Questions in Mixed-Initiative Conversational Search](https://mlsanity.com/paper/2402.07742/) (2024)
- [AGENT-CQ: Automatic Generation and Evaluation of Clarifying Questions for Conversational Search with LLMs](https://mlsanity.com/paper/2410.19692/) (2024)
- [Zero-Shot and Efficient Clarification Need Prediction in Conversational Search](https://mlsanity.com/paper/2503.00179/) (2025)
- [IDRBench: Benchmarking the Interactive Capabilities of Deep Research Agents](https://mlsanity.com/paper/2601.06676/) (2026)
- [Clarify-Then-Search: A Clarification Benchmark for Deep Search with End-to-End Nugget Restoration](https://mlsanity.com/paper/2608.20357/) (2026)

## Abstract (arXiv)

Asking clarification questions is an active area of research; however, resources for training and evaluating search clarification methods are not sufficient. To address this issue, we describe MIMICS-Duo, a new freely available dataset of 306 search queries with multiple clarifications (a total of 1,034 query-clarification pairs). MIMICS-Duo contains fine-grained annotations on clarification questions and their candidate answers and enhances the existing MIMICS datasets by enabling multi-dimensional evaluation of search clarification methods, including online and offline evaluation. We conduct extensive analysis to demonstrate the relationship between offline and online search clarification datasets and outline several research directions enabled by MIMICS-Duo. We believe that this resource will help researchers better understand clarification in search.