# An Interactive Query Generation Assistant using LLM-based Prompt Modification and User Feedback

arXiv: https://arxiv.org/abs/2311.11226 · HTML: https://mlsanity.com/paper/2311.11226/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 20 references that are in the map.
- First posted on arXiv 19 Nov 2023.

## Builds on (most cited first, 11 of 11)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021)
- [Document Expansion by Query Prediction](https://mlsanity.com/paper/1904.08375/) (2019)
- [Gradio: Hassle-Free Sharing and Testing of ML Models in the Wild](https://mlsanity.com/paper/1906.02569/) (2019)
- [ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning](https://mlsanity.com/paper/2111.10952/) (2021)
- [NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation](https://mlsanity.com/paper/2112.02721/) (2021)
- [Transfer Learning Approaches for Building Cross-Language Dense Retrieval Models](https://mlsanity.com/paper/2201.08471/) (2022)
- [LOME: Large Ontology Multilingual Extraction](https://mlsanity.com/paper/2101.12175/) (2021)
- [Unsupervised Document Expansion for Information Retrieval with Stochastic Text Generation](https://mlsanity.com/paper/2105.00666/) (2021)

## Built on by (most cited first, 6 of 6)

- [LLM Hallucinations in Practical Code Generation: Phenomena, Mechanism, and Mitigation](https://mlsanity.com/paper/2409.20550/) (2024)
- [Interactive AI with Retrieval-Augmented Generation for Next Generation Networking](https://mlsanity.com/paper/2401.11391/) (2024)
- [Generative Query Reformulation Using Ensemble Prompting, Document Fusion, and Relevance Feedback](https://mlsanity.com/paper/2405.17658/) (2024)
- [ConQRet: Benchmarking Fine-Grained Evaluation of Retrieval Augmented Argumentation with LLM Judges](https://mlsanity.com/paper/2412.05206/) (2024)
- [Query Understanding in LLM-based Conversational Information Seeking](https://mlsanity.com/paper/2504.06356/) (2025)
- [MARCO: Meta-Reflection with Cross-Referencing for Code Reasoning](https://mlsanity.com/paper/2505.17481/) (2025)

## Abstract (arXiv)

While search is the predominant method of accessing information, formulating effective queries remains a challenging task, especially for situations where the users are not familiar with a domain, or searching for documents in other languages, or looking for complex information such as events, which are not easily expressible as queries. Providing example documents or passages of interest, might be easier for a user, however, such query-by-example scenarios are prone to concept drift, and are highly sensitive to the query generation method. This demo illustrates complementary approaches of using LLMs interactively, assisting and enabling the user to provide edits and feedback at all stages of the query formulation process. The proposed Query Generation Assistant is a novel search interface which supports automatic and interactive query generation over a mono-linguial or multi-lingual document collection. Specifically, the proposed assistive interface enables the users to refine the queries generated by different LLMs, to provide feedback on the retrieved documents or passages, and is able to incorporate the users' feedback as prompts to generate more effective queries. The proposed interface is a valuable experimental tool for exploring fine-tuning and prompting of LLMs for query generation to qualitatively evaluate the effectiveness of retrieval and ranking models, and for conducting Human-in-the-Loop (HITL) experiments for complex search tasks where users struggle to formulate queries without such assistance.