# Can Generative LLMs Create Query Variants for Test Collections? An Exploratory Study

arXiv: https://arxiv.org/abs/2501.17981 · HTML: https://mlsanity.com/paper/2501.17981/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jan 2025.

## Built on by (most cited first, 10 of 10)

- [Beyond Contrastive Learning: Synthetic Data Enables List-wise Training with Multiple Levels of Relevance](https://mlsanity.com/paper/2503.23239/) (2025)
- [WisPaper: Your AI Scholar Search Engine](https://mlsanity.com/paper/2512.06879/) (2025)
- [DisastIR: A Comprehensive Information Retrieval Benchmark for Disaster Management](https://mlsanity.com/paper/2505.15856/) (2025)
- [Question-to-Knowledge (Q2K): Multi-Agent Generation of Inspectable Facts for Product Mapping](https://mlsanity.com/paper/2509.01182/) (2025)
- [QueStER: Query Specification for Generative keyword-based Retrieval](https://mlsanity.com/paper/2511.05301/) (2025)
- [LLM Ensemble for RAG: Role of Context Length in Zero-Shot Question Answering for BioASQ Challenge](https://mlsanity.com/paper/2509.08596/) (2025)
- [Trove: A Flexible Toolkit for Dense Retrieval](https://mlsanity.com/paper/2511.01857/) (2025)
- [RAQG-QPP: Query Performance Prediction with Retrieved Query Variants and Retrieval Augmented Query Generation](https://mlsanity.com/paper/2604.27244/) (2026)
- [STORM: Stepwise Token Optimization with Reward-Guided Beam Search](https://mlsanity.com/paper/2606.10621/) (2026)
- [The "Curse of Knowledge" in LLM Query Simulation: Concept Provenance for Tracing Answer-Side Intrusion](https://mlsanity.com/paper/2608.25245/) (2026)

## Abstract (arXiv)

This paper explores the utility of a Large Language Model (LLM) to automatically generate queries and query variants from a description of an information need. Given a set of information needs described as backstories, we explore how similar the queries generated by the LLM are to those generated by humans. We quantify the similarity using different metrics and examine how the use of each set would contribute to document pooling when building test collections. Our results show potential in using LLMs to generate query variants. While they may not fully capture the wide variety of human-generated variants, they generate similar sets of relevant documents, reaching up to 71.1% overlap at a pool depth of 100.