Leveraging LLMs in Scholarly Knowledge Graph Question Answering
Not on the map. Why?
Abstract (arXiv)
This paper presents a scholarly Knowledge Graph Question Answering (KGQA) that answers bibliographic natural language questions by leveraging a large language model (LLM) in a few-shot manner. The model initially identifies the top-n similar training questions related to a given test question via a BERT-based sentence encoder and retrieves their corresponding SPARQL. Using the top-n similar question-SPARQL pairs as an example and the test question creates a prompt. Then pass the prompt to the LLM and generate a SPARQL. Finally, runs the SPARQL against the underlying KG - ORKG (Open Research KG) endpoint and returns an answer. Our system achieves an F1 score of 99.0%, on SciQA - one of the Scholarly-QALD-23 challenge benchmarks.
Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 9 of its 18 references that are in the map. First posted on arXiv 16 Nov 2023.
Builds on
9 of 18 references.
References · 9
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
- 2022Large Language Models are few(1)-shot Table Reasonerscited by 116
- 2023Evaluating Open-Domain Question Answering in the Era of Large Language Modelscited by 111
- 2023DBLP-QuAD: A Question Answering Dataset over the DBLP Scholarly Knowledge Graphcited by 18
- 2020Question Answering on Scholarly Knowledge Graphscited by 7
- 2023GETT-QA: Graph Embedding based T2T Transformer for Knowledge Graph Question Answeringcited by 6
Built on by
Citing papers per year: 2023: 1; 2024: 5; 2025: 6; 2026: 1.
13 papers.
Sort:
By area:
- 2023Are Large Language Models Good Fact Checkers: A Preliminary Studycited by 8
- 2025CG-RAG: Research Question Answering by Citation Graph Retrieval-Augmented LLMscited by 8
- 2024SPARQL Generation with Entity Pre-trained GPT for KG Question Answeringcited by 6
- 2024Way to Specialist: Closing Loop Between Specialized LLM and Evolving Domain Knowledge Graphcited by 6
- 2025FIRESPARQL: A LLM-based Framework for SPARQL Query Generation over Scholarly Knowledge Graphscited by 5
- 2025ARUQULA -- An LLM based Text2SPARQL Approach using ReAct and Knowledge Graph Exploration Utilitiescited by 3
- 2024Scholarly Question Answering using Large Language Models in the NFDI4DataScience Gatewaycited by 2
- 2024Contri(e)ve: Context + Retrieve for Scholarly Question Answeringcited by 2
- 2025KERL: Knowledge-Enhanced Personalized Recipe Recommendation using Large Language Modelscited by 2
- 2025Enhancing Manufacturing Knowledge Access with LLMs and Context-aware Promptingcited by 2
- 2024PKG API: A Tool for Personal Knowledge Graph Managementcited by 1
- 2025How do Scaling Laws Apply to Knowledge Graph Engineering Tasks? The Impact of Model Size on Large Language Model Performancecited by 1
1 more
- 2026Reading Between the Citations: A Typed Claim Network for Scientific Literature29 May 2026
- 2025ARUQULA -- An LLM based Text2SPARQL Approach using ReAct and Knowledge Graph Exploration Utilities2 Oct 2025
- 2025FIRESPARQL: A LLM-based Framework for SPARQL Query Generation over Scholarly Knowledge Graphs14 Aug 2025
- 2025Enhancing Manufacturing Knowledge Access with LLMs and Context-aware Prompting30 Jul 2025
- 2025How do Scaling Laws Apply to Knowledge Graph Engineering Tasks? The Impact of Model Size on Large Language Model Performance22 May 2025
- 2025KERL: Knowledge-Enhanced Personalized Recipe Recommendation using Large Language Models20 May 2025
- 2025CG-RAG: Research Question Answering by Citation Graph Retrieval-Augmented LLMs25 Jan 2025
- 2024Way to Specialist: Closing Loop Between Specialized LLM and Evolving Domain Knowledge Graph28 Nov 2024
- 2024Contri(e)ve: Context + Retrieve for Scholarly Question Answering13 Sep 2024
- 2024Scholarly Question Answering using Large Language Models in the NFDI4DataScience Gateway11 Jun 2024
- 2024PKG API: A Tool for Personal Knowledge Graph Management12 Feb 2024
- 2024SPARQL Generation with Entity Pre-trained GPT for KG Question Answering1 Feb 2024
1 more
- 2024SPARQL Generation with Entity Pre-trained GPT for KG Question Answeringcited by 6
- 2024Way to Specialist: Closing Loop Between Specialized LLM and Evolving Domain Knowledge Graphcited by 6
- 2025FIRESPARQL: A LLM-based Framework for SPARQL Query Generation over Scholarly Knowledge Graphscited by 5
- 2025ARUQULA -- An LLM based Text2SPARQL Approach using ReAct and Knowledge Graph Exploration Utilitiescited by 3
- 2024Contri(e)ve: Context + Retrieve for Scholarly Question Answeringcited by 2
- 2025KERL: Knowledge-Enhanced Personalized Recipe Recommendation using Large Language Modelscited by 2
- 2025Enhancing Manufacturing Knowledge Access with LLMs and Context-aware Promptingcited by 2
- 2024PKG API: A Tool for Personal Knowledge Graph Managementcited by 1
- 2025How do Scaling Laws Apply to Knowledge Graph Engineering Tasks? The Impact of Model Size on Large Language Model Performancecited by 1
- 2026Reading Between the Citations: A Typed Claim Network for Scientific Literaturecited by 0
- 2026Reading Between the Citations: A Typed Claim Network for Scientific Literature29 May 2026
- 2025ARUQULA -- An LLM based Text2SPARQL Approach using ReAct and Knowledge Graph Exploration Utilities2 Oct 2025
- 2025FIRESPARQL: A LLM-based Framework for SPARQL Query Generation over Scholarly Knowledge Graphs14 Aug 2025
- 2025Enhancing Manufacturing Knowledge Access with LLMs and Context-aware Prompting30 Jul 2025
- 2025How do Scaling Laws Apply to Knowledge Graph Engineering Tasks? The Impact of Model Size on Large Language Model Performance22 May 2025
- 2025KERL: Knowledge-Enhanced Personalized Recipe Recommendation using Large Language Models20 May 2025
- 2024Way to Specialist: Closing Loop Between Specialized LLM and Evolving Domain Knowledge Graph28 Nov 2024
- 2024Contri(e)ve: Context + Retrieve for Scholarly Question Answering13 Sep 2024
- 2024PKG API: A Tool for Personal Knowledge Graph Management12 Feb 2024
- 2024SPARQL Generation with Entity Pre-trained GPT for KG Question Answering1 Feb 2024