# Towards Understanding Bias in Synthetic Data for Evaluation

arXiv: https://arxiv.org/abs/2506.10301 · HTML: https://mlsanity.com/paper/2506.10301/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 8 of its 23 references that are in the map.
- First posted on arXiv 12 Jun 2025.

## Placed in

- Synthetic data and self-improvement for LLMs / [LLM synthetic data generation](https://mlsanity.com/synth/llm-synthetic-data-generation/)

## Builds on (most cited first, 8 of 8)

- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [Is ChatGPT a Good NLG Evaluator? A Preliminary Study](https://mlsanity.com/paper/2303.04048/) (2023)
- [Large language models can accurately predict searcher preferences](https://mlsanity.com/paper/2309.10621/) (2023)
- [LLMs as Narcissistic Evaluators: When Ego Inflates Evaluation Scores](https://mlsanity.com/paper/2311.09766/) (2023)
- [Perspectives on Large Language Models for Relevance Judgment](https://mlsanity.com/paper/2304.09161/) (2023)
- [Synthetic Test Collections for Retrieval Evaluation](https://mlsanity.com/paper/2405.07767/) (2024)
- [SynDL: A Large-Scale Synthetic Test Collection for Passage Retrieval](https://mlsanity.com/paper/2408.16312/) (2024)
- [Report on the 1st Workshop on Large Language Model for Evaluation in Information Retrieval (LLM4Eval 2024) at SIGIR 2024](https://mlsanity.com/paper/2408.05388/) (2024)

## Built on by (most cited first, 7 of 7)

- [Query-Document Dense Vectors for LLM Relevance Judgment Bias Analysis](https://mlsanity.com/paper/2601.01751/) (2026)
- [AFRILANGTUTOR: Advancing Language Tutoring and Culture Education in Low-Resource Languages with Large Language Models](https://mlsanity.com/paper/2604.20996/) (2026)
- [DiffusionGS: Generative Search with Query Conditioned Diffusion in Kuaishou](https://mlsanity.com/paper/2508.17754/) (2025)
- [RMIT-ADM+S at the MMU-RAG NeurIPS 2025 Competition](https://mlsanity.com/paper/2602.20735/) (2026)
- [NodeSynth: Socially Aligned Synthetic Data for AI Evaluation](https://mlsanity.com/paper/2605.14381/) (2026)
- [The Coverage Illusion: From Pre-retrieval Routing Failure to Post-retrieval Cascades in a Production RAG System](https://mlsanity.com/paper/2605.27220/) (2026)
- [Evaluating Language Models in Realistic Conversational Contexts](https://mlsanity.com/paper/2608.26131/) (2026)

## Abstract (arXiv)

Test collections are crucial for evaluating Information Retrieval (IR) systems. Creating a diverse set of user queries for these collections can be challenging, and obtaining relevance judgments, which indicate how well retrieved documents match a query, is often costly and resource-intensive. Recently, generating synthetic datasets using Large Language Models (LLMs) has gained attention in various applications. While previous work has used LLMs to generate synthetic queries or documents to improve ranking models, using LLMs to create synthetic test collections is still relatively unexplored. Previous work~citerahmani2024synthetic showed that synthetic test collections have the potential to be used for system evaluation, however, more analysis is needed to validate this claim. In this paper, we thoroughly investigate the reliability of synthetic test collections constructed using LLMs, where LLMs are used to generate synthetic queries, labels, or both. In particular, we examine the potential biases that might occur when such test collections are used for evaluation. We first empirically show the presence of such bias in evaluation results and analyse the effects it might have on system evaluation. We further validate the presence of such bias using a linear mixed-effects model. Our analysis shows that while the effect of bias present in evaluation results obtained using synthetic test collections could be significant, for e.g.~computing absolute system performance, its effect may not be as significant in comparing relative system performance. Codes and data are available at: https://github.com/rahmanidashti/BiasSyntheticData.