# Sanitizing Synthetic Training Data Generation for Question Answering over Knowledge Graphs

arXiv: https://arxiv.org/abs/2009.04915 · HTML: https://mlsanity.com/paper/2009.04915/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Sep 2020.

## Built on by (most cited first, 4 of 4)

- [FVQA 2.0: Introducing Adversarial Samples into Fact-based Visual Question Answering](https://mlsanity.com/paper/2303.10699/) (2023)
- [Spider4SPARQL: A Complex Benchmark for Evaluating Knowledge Graph Question Answering Systems](https://mlsanity.com/paper/2309.16248/) (2023)
- [Would You Ask it that Way? Measuring and Improving Question Naturalness for Knowledge Graph Question Answering](https://mlsanity.com/paper/2205.12768/) (2022)
- [PDPK: A Framework to Synthesise Process Data and Corresponding Procedural Knowledge for Manufacturing](https://mlsanity.com/paper/2308.08371/) (2023)

## Abstract (arXiv)

Synthetic data generation is important to training and evaluating neural models for question answering over knowledge graphs. The quality of the data and the partitioning of the datasets into training, validation and test splits impact the performance of the models trained on this data. If the synthetic data generation depends on templates, as is the predominant approach for this task, there may be a leakage of information via a shared basis of templates across data splits if the partitioning is not performed hygienically. This paper investigates the extent of such information leakage across data splits, and the ability of trained models to generalize to test data when the leakage is controlled. We find that information leakage indeed occurs and that it affects performance. At the same time, the trained models do generalize to test data under the sanitized partitioning presented here. Importantly, these findings extend beyond the particular flavor of question answering task we studied and raise a series of difficult questions around template-based synthetic data generation that will necessitate additional research.