# Improving Content Retrievability in Search with Controllable Query Generation

arXiv: https://arxiv.org/abs/2303.11648 · HTML: https://mlsanity.com/paper/2303.11648/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 67 references that are in the map.
- First posted on arXiv 21 Mar 2023.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://mlsanity.com/paper/2104.08663/) (2021)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [Pretrained Transformers for Text Ranking: BERT and Beyond](https://mlsanity.com/paper/2010.06467/) (2020)
- [Promptagator: Few-shot Dense Retrieval From 8 Examples](https://mlsanity.com/paper/2209.11755/) (2022)
- [Document Expansion by Query Prediction](https://mlsanity.com/paper/1904.08375/) (2019)
- [GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval](https://mlsanity.com/paper/2112.07577/) (2021)
- [Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generation](https://mlsanity.com/paper/2206.10128/) (2022)
- [Evaluating the Robustness of Retrieval Pipelines with Query Variation Generators](https://mlsanity.com/paper/2111.13057/) (2021)
- [Declarative Experimentation in Information Retrieval using PyTerrier](https://mlsanity.com/paper/2007.14271/) (2020)
- [Embedding-based Zero-shot Retrieval through Query Generation](https://mlsanity.com/paper/2009.10270/) (2020)
- [TREC 2020 Podcasts Track Overview](https://mlsanity.com/paper/2103.15953/) (2021)
- [One Deep Music Representation to Rule Them All? : A comparative analysis of different representation learning strategies](https://mlsanity.com/paper/1802.04051/) (2018)
- [Boosting Search Performance Using Query Variations](https://mlsanity.com/paper/1811.06147/) (2018)
- [Exposing Query Identification for Search Transparency](https://mlsanity.com/paper/2110.07701/) (2021)
- [Discovering Latent Patterns from the Analysis of User-Curated Movie Lists](https://mlsanity.com/paper/1308.5125/) (2013)
- [Studying Retrievability of Publications and Datasets in an Integrated Retrieval System](https://mlsanity.com/paper/2205.00937/) (2022)

## Built on by (most cited first, 6 of 6)

- [Large Language Model Augmented Narrative Driven Recommendations](https://mlsanity.com/paper/2306.02250/) (2023)
- [Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspective](https://mlsanity.com/paper/2407.06992/) (2024)
- [Query Understanding in the Age of Large Language Models](https://mlsanity.com/paper/2306.16004/) (2023)
- [Generative Information Retrieval Evaluation](https://mlsanity.com/paper/2404.08137/) (2024)
- [Improving Scientific Document Retrieval with Concept Coverage-based Query Set Generation](https://mlsanity.com/paper/2502.11181/) (2025)
- [IR2: Information Regularization for Information Retrieval](https://mlsanity.com/paper/2402.16200/) (2024)

## Abstract (arXiv)

An important goal of online platforms is to enable content discovery, i.e. allow users to find a catalog entity they were not familiar with. A pre-requisite to discover an entity, e.g. a book, with a search engine is that the entity is retrievable, i.e. there are queries for which the system will surface such entity in the top results. However, machine-learned search engines have a high retrievability bias, where the majority of the queries return the same entities. This happens partly due to the predominance of narrow intent queries, where users create queries using the title of an already known entity, e.g. in book search 'harry potter'. The amount of broad queries where users want to discover new entities, e.g. in music search 'chill lyrical electronica with an atmospheric feeling to it', and have a higher tolerance to what they might find, is small in comparison. We focus here on two factors that have a negative impact on the retrievability of the entities (I) the training data used for dense retrieval models and (II) the distribution of narrow and broad intent queries issued in the system. We propose CtrlQGen, a method that generates queries for a chosen underlying intent-narrow or broad. We can use CtrlQGen to improve factor (I) by generating training data for dense retrieval models comprised of diverse synthetic queries. CtrlQGen can also be used to deal with factor (II) by suggesting queries with broader intents to users. Our results on datasets from the domains of music, podcasts, and books reveal that we can significantly decrease the retrievability bias of a dense retrieval model when using CtrlQGen. First, by using the generated queries as training data for dense models we make 9% of the entities retrievable (go from zero to non-zero retrievability). Second, by suggesting broader queries to users, we can make 12% of the entities retrievable in the best case.