mlsanityData to 14 Oct 2026

Aligning Large Language Models to a Domain-specific Graph Database for NL2GQL

arXiv 2402.16567 ↗First posted 26 Feb 2024cs.CL cs.AI cs.DBcited by 14

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

Graph Databases (Graph DB) find extensive application across diverse domains such as finance, social networks, and medicine. Yet, the translation of Natural Language (NL) into the Graph Query Language (GQL), referred to as NL2GQL, poses significant challenges owing to its intricate and specialized nature. Some approaches have sought to utilize Large Language Models (LLMs) to address analogous tasks like text2SQL. Nonetheless, in the realm of NL2GQL tasks tailored to a particular domain, the absence of domain-specific NL-GQL data pairs adds complexity to aligning LLMs with the graph DB. To tackle this challenge, we present a well-defined pipeline. Initially, we utilize ChatGPT to generate NL-GQL data pairs, leveraging the provided graph DB with self-instruction. Subsequently, we employ the generated data to fine-tune LLMs, ensuring alignment between LLMs and the graph DB. Moreover, we find the importance of relevant schema in efficiently generating accurate GQLs. Thus, we introduce a method to extract relevant schema as the input context. We evaluate our method using two carefully constructed datasets derived from graph DBs in the finance and medicine domains, named FinGQL and MediGQL. Experimental results reveal that our approach significantly outperforms a set of baseline methods, with improvements of 5.90 and 6.36 absolute points on EM, and 6.00 and 7.09 absolute points on EX for FinGQL and MediGQL, respectively.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 32 of its 50 references that are in the map. First posted on arXiv 26 Feb 2024.

Builds on

32 of 50 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructions20 Dec

Other references · 31

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  4. 2023Qwen Technical Reportcited by 3,344
  5. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  6. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,818
  7. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  8. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  9. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  10. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 896
  11. 2022A Survey on In-context Learningcited by 721
  12. 2023Baichuan 2: Open Large-scale Language Modelscited by 670
19 more
  1. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 668
  2. 2021Editing Factual Knowledge in Language Modelscited by 553
  3. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 477
  4. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 463
  5. 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 385
  6. 2020Multilingual Translation with Extensible Multilingual Pretraining and Finetuningcited by 287
  7. 2022Unified Structure Generation for Universal Information Extractioncited by 221
  8. 2020Beyond I.I.D.: Three Levels of Generalization for Question Answering on Knowledge Basescited by 171
  9. 2023Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Modelscited by 148
  10. 2023BianQue: Balancing the Questioning and Suggestion Ability of Health LLMs with Multi-turn Health Conversations Polished by ChatGPTcited by 65
  11. 2023How to Prompt LLMs for Text-to-SQL: A Study in Zero-shot, Single-domain, and Cross-domain Settingscited by 58
  12. 2023Knowledge Graph-Augmented Language Models for Knowledge-Grounded Dialogue Generationcited by 54
  13. 2023KnowledGPT: Enhancing Large Language Models with Retrieval and Storage Access on Knowledge Basescited by 52
  14. 2023FlexKBQA: A Flexible LLM-Powered Framework for Few-Shot Knowledge Base Question Answeringcited by 51
  15. 2023LLM-assisted Knowledge Graph Engineering: Experiments with ChatGPTcited by 42
  16. 2019Demystifying Graph Databases: Analysis and Taxonomy of Data Organization, System Designs, and Graph Queriescited by 22
  17. 2021Knowledge-Aware Graph-Enhanced GPT-2 for Dialogue State Trackingcited by 20
  18. 2022Knowledge Graph Question Answering Datasets and Their Generalizability: Are They Enough for Future Research?cited by 13
  19. 2023R3-NL2GQL: A Model Coordination and Knowledge Graph Alignment Approach for NL2GQLcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 8; 2026: 2.

14 papers.

Sort:

By area:

  1. 2024A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challengescited by 90
  2. 2025A Survey of Graph Retrieval-Augmented Generation for Customized Large Language Modelscited by 88
  3. 2024CypherBench: Towards Precise Retrieval over Full-scale Modern Knowledge Graphs in the LLM Eracited by 16
  4. 2025LLM/Agent-as-Data-Analyst: A Surveycited by 16
  5. 2025Graph2text or Graph2token: A Perspective of Large Language Models for Graph Learningcited by 12
  6. 2024A Collaborative Multi-Agent Approach to Retrieval-Augmented Generation Across Diverse Datacited by 10
  7. 2025Synergizing Logical Reasoning, Knowledge Management and Collaboration in Multi-Agent LLM Systemcited by 10
  8. 2024NAT-NL2GQL: A Novel Multi-Agent Framework for Translating Natural Language to Graph Query Languagecited by 9
  9. 2025Enhancing Text2Cypher with Schema Filteringcited by 6
  10. 2025ZOGRASCOPE: A New Benchmark for Semantic Parsing over Property Graphscited by 1
  11. 2025TagRAG: Tag-guided Hierarchical Knowledge Graph Retrieval-Augmented Generationcited by 1
  12. 2026Achieving Precise Text-To-Cypher Via Grounded Knowledge Graph Data Generationcited by 1
2 more
  1. 2025Multi-turn Natural Language to Graph Query Language Translationcited by 0
  2. 2026Structurally Speaking: Motif-Oriented Graph Captioning through Bidirectional Graph-Text Translationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024GLaM: Fine-Tuning Large Language Models for Domain Knowledge Graph Alignment via Neighborhood Partitioning and Generative Subgraph Encoding
  2. 2026Toward Multi-Database Query Reasoning for Text2Cypher
  3. 2023Large Language Models on Graphs: A Comprehensive Survey
  4. 2024Towards Evaluating Large Language Models for Graph Query Generation
  5. 2024SyntheT2C: Generating Synthetic Data for Fine-Tuning Large Language Models on the Text2Cypher Task
  6. 2025ARUQULA -- An LLM based Text2SPARQL Approach using ReAct and Knowledge Graph Exploration Utilities
  7. 2025LLMSQL: Upgrading WikiSQL for the LLM Era of Text-to-SQL
  8. 2025Automating SPARQL Query Translations between DBpedia and Wikidata