# Knowledge Graph Question Answering for Materials Science (KGQA4MAT): Developing Natural Language Interface for Metal-Organic Frameworks Knowledge Graph (MOF-KG) Using LLM

arXiv: https://arxiv.org/abs/2309.11361 · HTML: https://mlsanity.com/paper/2309.11361/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 46 references that are in the map.
- First posted on arXiv 20 Sep 2023.

## Builds on (most cited first, 9 of 9)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022)
- [Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT](https://mlsanity.com/paper/2302.10198/) (2023)
- [How Robust is GPT-3.5 to Predecessors? A Comprehensive Study on Language Understanding Tasks](https://mlsanity.com/paper/2303.00293/) (2023)
- [ChatGPT versus Traditional Question Answering for Knowledge Graphs: Current Status and Future Directions Towards Knowledge Graph Chatbots](https://mlsanity.com/paper/2302.06466/) (2023)
- [A Comparative Survey of Recent Natural Language Interfaces for Databases](https://mlsanity.com/paper/1906.08990/) (2019)
- [Learning to Rank Query Graphs for Complex Question Answering over Knowledge Graphs](https://mlsanity.com/paper/1811.01118/) (2018)
- [An Ontology for the Materials Design Domain](https://mlsanity.com/paper/2006.07712/) (2020)
- [Building Open Knowledge Graph for Metal-Organic Frameworks (MOF-KG): Challenges and Case Studies](https://mlsanity.com/paper/2207.04502/) (2022)

## Built on by (most cited first, 5 of 5)

- [SPARQL Generation: an analysis on fine-tuning OpenLLaMA for Question Answering over a Life Science Knowledge Graph](https://mlsanity.com/paper/2402.04627/) (2024)
- [Construction and Application of Materials Knowledge Graph in Multidisciplinary Materials Science via Large Language Model](https://mlsanity.com/paper/2404.03080/) (2024)
- [Towards Agentic Intelligence for Materials Science](https://mlsanity.com/paper/2602.00169/) (2026)
- [Enhancing Manufacturing Knowledge Access with LLMs and Context-aware Prompting](https://mlsanity.com/paper/2507.22619/) (2025)
- [Mechanisms of Matter: Language Inferential Benchmark on Physicochemical Hypothesis in Materials Synthesis](https://mlsanity.com/paper/2509.25281/) (2025)

## Abstract (arXiv)

We present a comprehensive benchmark dataset for Knowledge Graph Question Answering in Materials Science (KGQA4MAT), with a focus on metal-organic frameworks (MOFs). A knowledge graph for metal-organic frameworks (MOF-KG) has been constructed by integrating structured databases and knowledge extracted from the literature. To enhance MOF-KG accessibility for domain experts, we aim to develop a natural language interface for querying the knowledge graph. We have developed a benchmark comprised of 161 complex questions involving comparison, aggregation, and complicated graph structures. Each question is rephrased in three additional variations, resulting in 644 questions and 161 KG queries. To evaluate the benchmark, we have developed a systematic approach for utilizing the LLM, ChatGPT, to translate natural language questions into formal KG queries. We also apply the approach to the well-known QALD-9 dataset, demonstrating ChatGPT's potential in addressing KGQA issues for different platforms and query languages. The benchmark and the proposed approach aim to stimulate further research and development of user-friendly and efficient interfaces for querying domain-specific materials science knowledge graphs, thereby accelerating the discovery of novel materials.