# The "Curse of Knowledge" in LLM Query Simulation: Concept Provenance for Tracing Answer-Side Intrusion

arXiv: https://arxiv.org/abs/2608.25245 · HTML: https://mlsanity.com/paper/2608.25245/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 48 references that are in the map.
- First posted on arXiv 26 Aug 2026.

## Builds on (most cited first, 13 of 13)

- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [Preference Leakage: A Contamination Problem in LLM-as-a-judge](https://mlsanity.com/paper/2502.01534/) (2025)
- [From Matching to Generation: A Survey on Generative Information Retrieval](https://mlsanity.com/paper/2404.14851/) (2024)
- [Neural Retrievers are Biased Towards LLM-Generated Content](https://mlsanity.com/paper/2310.20501/) (2023)
- [USimAgent: Large Language Models for Simulating Search Users](https://mlsanity.com/paper/2403.09142/) (2024)
- [User Simulation for Evaluating Information Access Systems](https://mlsanity.com/paper/2306.08550/) (2023)
- [BASES: Large-scale Web Search User Simulation with Large Language Model based Agents](https://mlsanity.com/paper/2402.17505/) (2024)
- [Synthetic Test Collections for Retrieval Evaluation](https://mlsanity.com/paper/2405.07767/) (2024)
- [Can Generative LLMs Create Query Variants for Test Collections? An Exploratory Study](https://mlsanity.com/paper/2501.17981/) (2025)
- [Temporal-Aware User Behaviour Simulation with Large Language Models for Recommender Systems](https://mlsanity.com/paper/2509.16895/) (2025)
- [Validating Search Query Simulations: A Taxonomy of Measures](https://mlsanity.com/paper/2601.11412/) (2026)
- [Verifiable User Simulation for Search and Recommendation Systems](https://mlsanity.com/paper/2606.14474/) (2026)
- [Demographically-Inspired Query Variants Using an LLM](https://mlsanity.com/paper/2508.17644/) (2025)

## Abstract (arXiv)

LLM-generated search queries are widely used to augment IR evaluation, yet they may contain concepts that presuppose answer-side document knowledge, violating the information-access boundary of pre-search users. Existing validation metrics, including overlap, diversity, and effectiveness, cannot distinguish rare human-tail variation from candidate answer-side intrusion. We introduce concept provenance, a framework that assigns query concepts to backstory-supported, human-central, human-tail, and candidate answer-side zones, operationalizing a boundary that retrieval metrics alone cannot detect. Applying concept provenance to 77,004 queries across 100 UQV100 topics, 8 LLMs, and 5 prompt conditions with two extraction pipelines, we obtain a cross-pipeline token-HCIR Spearman rho of 1.0 over five condition means. Candidate answer-side concepts constitute 7.40 percent of non-generic concepts and appear in 97 of 100 topics, with topic explaining approximately 67 percent of variance. Human validation yields 68.2 percent relaxed precision, revealing two mechanisms: knowledge intrusion at 45.5 percent and deployment intrusion at 45.0 percent. Diagnostic probes show disproportionate localized retrieval effects, with deletion effect size d = -0.47 compared with d = -0.34 for random deletion, but these concepts explain less than 2 percent of aggregate evaluation variance. Concept provenance therefore serves as a boundary-compliance diagnostic rather than an evaluation-shift predictor. Under the tested conditions, no prompt condition eliminates intrusion; post-generation concept-provenance selection achieves 99 percent elimination.