# Q-NL Verifier: Leveraging Synthetic Data for Robust Knowledge Graph Question Answering

arXiv: https://arxiv.org/abs/2503.01385 · HTML: https://mlsanity.com/paper/2503.01385/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Synthetic reasoning data and bootstrapping, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 13 of its 28 references that are in the map.
- First posted on arXiv 3 Mar 2025.

## Placed in

- Synthetic data and self-improvement for LLMs / [Synthetic reasoning data and bootstrapping](https://mlsanity.com/synth/synthetic-reasoning-data-and-bootstrapping/)

## Builds on (most cited first, 13 of 13)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://mlsanity.com/paper/2412.13663/) (2024)
- [Evaluation of OpenAI o1: Opportunities and Challenges of AGI](https://mlsanity.com/paper/2409.18486/) (2024)
- [The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Models](https://mlsanity.com/paper/2401.05618/) (2024)
- [FlexKBQA: A Flexible LLM-Powered Framework for Few-Shot Knowledge Base Question Answering](https://mlsanity.com/paper/2308.12060/) (2023)
- [Neural Machine Translating from Natural Language to SPARQL](https://mlsanity.com/paper/1906.09302/) (2019)
- [SPARQL Generation: an analysis on fine-tuning OpenLLaMA for Question Answering over a Life Science Knowledge Graph](https://mlsanity.com/paper/2402.04627/) (2024)
- [A Comprehensive Evaluation of Neural SPARQL Query Generation from Natural Language Questions](https://mlsanity.com/paper/2304.07772/) (2023)
- [Knowledge Graph Question Answering using Graph-Pattern Isomorphism](https://mlsanity.com/paper/2103.06752/) (2021)
- [A Holistic Natural Language Generation Framework for the Semantic Web](https://mlsanity.com/paper/1911.01248/) (2019)
- [NABU - Multilingual Graph-based Neural RDF Verbalizer](https://mlsanity.com/paper/2009.07728/) (2020)

## Built on by (most cited first, 2 of 2)

- [The benefits of query-based KGQA systems for complex and temporal questions in LLM era](https://mlsanity.com/paper/2507.11954/) (2025)
- [GraphSeek: Next-Generation Graph Analytics with LLMs](https://mlsanity.com/paper/2602.11052/) (2026)

## Abstract (arXiv)

Question answering (QA) requires accurately aligning user questions with structured queries, a process often limited by the scarcity of high-quality query-natural language (Q-NL) pairs. To overcome this, we present Q-NL Verifier, an approach to generating high-quality synthetic pairs of queries and NL translations. Our approach relies on large language models (LLMs) to generate semantically precise natural language paraphrases of structured queries. Building on these synthetic Q-NL pairs, we introduce a learned verifier component that automatically determines whether a generated paraphrase is semantically equivalent to the original query. Our experiments with the well-known LC-QuAD 2.0 benchmark show that Q-NL Verifier generalizes well to paraphrases from other models and even human-authored translations. Our approach strongly aligns with human judgments across varying query complexities and outperforms existing NLP metrics in assessing semantic correctness. We also integrate the verifier into QA pipelines, showing that verifier-filtered synthetic data has significantly higher quality in terms of translation correctness and enhances NL to Q translation accuracy. Lastly, we release an updated version of the LC-QuAD 2.0 benchmark containing our synthetic Q-NL pairs and verifier scores, offering a new resource for robust and scalable QA.