# SPARQL Generation with Entity Pre-trained GPT for KG Question Answering

arXiv: https://arxiv.org/abs/2402.00969 · HTML: https://mlsanity.com/paper/2402.00969/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 15 references that are in the map.
- First posted on arXiv 1 Feb 2024.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Knowledge Graphs](https://mlsanity.com/paper/2003.02320/) (2020)
- [A Survey on Transfer Learning in Natural Language Processing](https://mlsanity.com/paper/2007.04239/) (2020)
- [Leveraging LLMs in Scholarly Knowledge Graph Question Answering](https://mlsanity.com/paper/2311.09841/) (2023)
- [Induced Natural Language Rationales and Interleaved Markup Tokens Enable Extrapolation in Large Language Models](https://mlsanity.com/paper/2208.11445/) (2022)
- [NLQxform: A Language Model-based Question to SPARQL Transformer](https://mlsanity.com/paper/2311.07588/) (2023)
- [DBLPLink: An Entity Linker for the DBLP Scholarly Knowledge Graph](https://mlsanity.com/paper/2309.07545/) (2023)
- [BERTologyNavigator: Advanced Question Answering with BERT-based Semantics](https://mlsanity.com/paper/2401.09553/) (2024)

## Built on by (most cited first, 6 of 6)

- [Assessing SPARQL capabilities of Large Language Models](https://mlsanity.com/paper/2409.05925/) (2024)
- [Leveraging small language models for Text2SPARQL tasks to improve the resilience of AI assistance](https://mlsanity.com/paper/2405.17076/) (2024)
- [Thinking with Knowledge Graphs: Enhancing LLM Reasoning Through Structured Data](https://mlsanity.com/paper/2412.10654/) (2024)
- [FIRESPARQL: A LLM-based Framework for SPARQL Query Generation over Scholarly Knowledge Graphs](https://mlsanity.com/paper/2508.10467/) (2025)
- [InteracSPARQL: An Interactive System for SPARQL Query Refinement Using Natural Language Explanations](https://mlsanity.com/paper/2511.02002/) (2025)
- [Beyond Single Pass, Looping Through Time: KG-IRAG with Iterative Knowledge Retrieval](https://mlsanity.com/paper/2503.14234/) (2025)

## Abstract (arXiv)

Knowledge Graphs popularity has been rapidly growing in last years. All that knowledge is available for people to query it through the many online databases on the internet. Though, it would be a great achievement if non-programmer users could access whatever information they want to know. There has been a lot of effort oriented to solve this task using natural language processing tools and creativity encouragement by way of many challenges. Our approach focuses on assuming a correct entity linking on the natural language questions and training a GPT model to create SPARQL queries from them. We managed to isolate which property of the task can be the most difficult to solve at few or zero-shot and we proposed pre-training on all entities (under CWA) to improve the performance. We obtained a 62.703% accuracy of exact SPARQL matches on testing at 3-shots, a F1 of 0.809 on the entity linking challenge and a F1 of 0.009 on the question answering challenge.