# SPARQL Generation: an analysis on fine-tuning OpenLLaMA for Question Answering over a Life Science Knowledge Graph

arXiv: https://arxiv.org/abs/2402.04627 · HTML: https://mlsanity.com/paper/2402.04627/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 29 references that are in the map.
- First posted on arXiv 7 Feb 2024.

## Builds on (most cited first, 15 of 15)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [KQA Pro: A Dataset with Explicit Compositional Programs for Complex Question Answering over Knowledge Base](https://mlsanity.com/paper/2007.03875/) (2020)
- [Language Models Can Teach Themselves to Program Better](https://mlsanity.com/paper/2207.14502/) (2022)
- [On the Potential of Artificial Intelligence Chatbots for Data Exploration of Federated Bioinformatics Knowledge Graphs](https://mlsanity.com/paper/2304.10427/) (2023)
- [Bio-SODA: Enabling Natural Language Question Answering over Knowledge Graphs without Training Data](https://mlsanity.com/paper/2104.13744/) (2021)
- [Knowledge Graph Question Answering for Materials Science (KGQA4MAT): Developing Natural Language Interface for Metal-Organic Frameworks Knowledge Graph (MOF-KG) Using LLM](https://mlsanity.com/paper/2309.11361/) (2023)
- [Lessons learned to boost a bioinformatics knowledge base reusability, the Bgee experience](https://mlsanity.com/paper/2303.12329/) (2023)

## Built on by (most cited first, 19 of 19)

- [DALK: Dynamic Co-Augmentation of LLMs and KG to answer Alzheimer's Disease Questions with Scientific Literature](https://mlsanity.com/paper/2405.04819/) (2024)
- [Assessing SPARQL capabilities of Large Language Models](https://mlsanity.com/paper/2409.05925/) (2024)
- [Leveraging small language models for Text2SPARQL tasks to improve the resilience of AI assistance](https://mlsanity.com/paper/2405.17076/) (2024)
- [LinkQ: An LLM-Assisted Visual Interface for Knowledge Graph Question-Answering](https://mlsanity.com/paper/2406.06621/) (2024)
- [Thinking with Knowledge Graphs: Enhancing LLM Reasoning Through Structured Data](https://mlsanity.com/paper/2412.10654/) (2024)
- [SPARQL Query Generation with LLMs: Measuring the Impact of Training Data Memorization and Knowledge Injection](https://mlsanity.com/paper/2507.13859/) (2025)
- [Bioinformatics and Biomedical Informatics with ChatGPT: Year One Review](https://mlsanity.com/paper/2403.15274/) (2024)
- [OntologyRAG: Better and Faster Biomedical Code Mapping with Retrieval-Augmented Generation (RAG) Leveraging Ontology Knowledge Graphs and Large Language Models](https://mlsanity.com/paper/2502.18992/) (2025)
- [Q-NL Verifier: Leveraging Synthetic Data for Robust Knowledge Graph Question Answering](https://mlsanity.com/paper/2503.01385/) (2025)
- [KERL: Knowledge-Enhanced Personalized Recipe Recommendation using Large Language Models](https://mlsanity.com/paper/2505.14629/) (2025)
- [Enhancing Manufacturing Knowledge Access with LLMs and Context-aware Prompting](https://mlsanity.com/paper/2507.22619/) (2025)
- [InteracSPARQL: An Interactive System for SPARQL Query Refinement Using Natural Language Explanations](https://mlsanity.com/paper/2511.02002/) (2025)
- [RAGONITE: Iterative Retrieval on Induced Databases and Verbalized RDF for Conversational QA over KGs with RAG](https://mlsanity.com/paper/2412.17690/) (2024)
- [The benefits of query-based KGQA systems for complex and temporal questions in LLM era](https://mlsanity.com/paper/2507.11954/) (2025)
- [GraphSeek: Next-Generation Graph Analytics with LLMs](https://mlsanity.com/paper/2602.11052/) (2026)
- [The Role of Visualization in LLM-Assisted Knowledge Graph Systems: Effects on User Trust, Exploration, and Workflows](https://mlsanity.com/paper/2505.21512/) (2025)
- [MARS: Multi-hop Adaptive Retrieval and SPARQL Generation for KGQA](https://mlsanity.com/paper/2607.14561/) (2026)
- [KGVoyager: Knowledge Graph Agnostic Question Answering via Agentic Navigation](https://mlsanity.com/paper/2609.01780/) (2026)
- [Enabling Knowledge Graph Understanding at Scale with the EXplore Your Graphs ENgine (EXYGEN)](https://mlsanity.com/paper/2609.11569/) (2026)

## Abstract (arXiv)

The recent success of Large Language Models (LLM) in a wide range of Natural Language Processing applications opens the path towards novel Question Answering Systems over Knowledge Graphs leveraging LLMs. However, one of the main obstacles preventing their implementation is the scarcity of training data for the task of translating questions into corresponding SPARQL queries, particularly in the case of domain-specific KGs. To overcome this challenge, in this study, we evaluate several strategies for fine-tuning the OpenLlama LLM for question answering over life science knowledge graphs. In particular, we propose an end-to-end data augmentation approach for extending a set of existing queries over a given knowledge graph towards a larger dataset of semantically enriched question-to-SPARQL query pairs, enabling fine-tuning even for datasets where these pairs are scarce. In this context, we also investigate the role of semantic "clues" in the queries, such as meaningful variable names and inline comments. Finally, we evaluate our approach over the real-world Bgee gene expression knowledge graph and we show that semantic clues can improve model performance by up to 33% compared to a baseline with random variable names and no comments included.