# Text2GraphQuery-Bench: A Text to Graph Query Benchmark

arXiv: https://arxiv.org/abs/2602.11745 · HTML: https://mlsanity.com/paper/2602.11745/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 59 references that are in the map.
- First posted on arXiv 12 Feb 2026.

## Builds on (most cited first, 18 of 18)

- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs](https://mlsanity.com/paper/2305.03111/) (2023)
- [Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows](https://mlsanity.com/paper/2411.07763/) (2024)
- [The Dawn of Natural Language to SQL: Are We Fully Ready?](https://mlsanity.com/paper/2406.01265/) (2024)
- [Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content](https://mlsanity.com/paper/2110.04406/) (2021)
- [MFTCoder: Boosting Code LLMs with Multitask Fine-Tuning](https://mlsanity.com/paper/2311.02303/) (2023)
- [A Survey of Data Agents: Emerging Paradigm or Overstated Hype?](https://mlsanity.com/paper/2510.23587/) (2025)
- [DeepEye-SQL: A Software-Engineering-Inspired Text-to-SQL Framework](https://mlsanity.com/paper/2510.17586/) (2025)
- [SPINACH: SPARQL-Based Information Navigation for Challenging Real-World Questions](https://mlsanity.com/paper/2407.11417/) (2024)
- [Agentar-Scale-SQL: Advancing Text-to-SQL through Orchestrated Test-Time Scaling](https://mlsanity.com/paper/2509.24403/) (2025)
- [SyntheT2C: Generating Synthetic Data for Fine-Tuning Large Language Models on the Text2Cypher Task](https://mlsanity.com/paper/2406.10710/) (2024)
- [CypherBench: Towards Precise Retrieval over Full-scale Modern Knowledge Graphs in the LLM Era](https://mlsanity.com/paper/2412.18702/) (2024)
- [Enhancing Supply Chain Visibility with Knowledge Graphs and Large Language Models](https://mlsanity.com/paper/2408.07705/) (2024)
- [BYOKG-RAG: Multi-Strategy Graph Retrieval for Knowledge Graph Question Answering](https://mlsanity.com/paper/2507.04127/) (2025)
- [NAT-NL2GQL: A Novel Multi-Agent Framework for Translating Natural Language to Graph Query Language](https://mlsanity.com/paper/2412.10434/) (2024)
- [R3-NL2GQL: A Model Coordination and Knowledge Graph Alignment Approach for NL2GQL](https://mlsanity.com/paper/2311.01862/) (2023)
- [Graphiti: Bridging Graph and Relational Database Queries](https://mlsanity.com/paper/2504.03182/) (2025)

## Built on by (most cited first, 3 of 3)

- [Achieving Precise Text-To-Cypher Via Grounded Knowledge Graph Data Generation](https://mlsanity.com/paper/2606.14325/) (2026)
- [Toward Multi-Database Query Reasoning for Text2Cypher](https://mlsanity.com/paper/2605.10373/) (2026)
- [SEGRA: A Structured Experience Guided Reasoning Agent for Property Graph Question Answering](https://mlsanity.com/paper/2607.22713/) (2026)

## Abstract (arXiv)

Graph models are fundamental to data analysis in domains rich with complex relationships. Unlike SQL, which benefits from a rel- atively unified standard and widespread familiarity, graph query languages are diverse (e.g., Cypher, GQL, SQL/PGQ) and far less fa- miliar to most users, making them significantly harder to learn and use. Text-to-Graph-Query systems address this barrier by trans- lating natural language into executable graph queries, enabling LLMs to serve as interfaces for Graph Database Management Systems (GDBMS). Existing benchmarks are limited in language coverage, rely on rigid synthesis, and lack comprehensive evaluation. We present Text2GraphQuery-Bench, the first benchmark covering all mainstream declarative property graph query languages (Cypher, GQL, and SQL/PGQ). It contains 267,276 (Question, Graph Query) pairs across 34 databases and 13 domains. Its construction supports adaptation from heterogeneous resources and domain-aware synthesis, while its Graph-IR-based design enables rapid extension to new languages. The evaluation protocol reports Grammar, GLEU, Similarity, and EX under graph-native difficulty, question abstraction, and schema aliasing. Experiments on 8 LLMs reveal: (i) a significant language gap exists - zero-shot GQL and SQL/PGQ Grammar is far below Cypher, yet few-shot prompting largely recovers it; (ii) fine-tuning an 8B model reaches or exceeds zero-shot large models, indicating unfamiliarity - rather than model capacity - is the primary barrier; (iii) as supervision increases, syntax errors recede, shifting bottlenecks to aggregation logic in GQL and schema linking in SQL/PGQ; (iv) higher question abstraction degrades EX due to intent-to-schema grounding issues, while schema aliasing has minimal impact; (v) EX consistently degrades from Easy to Extra Hard, with Extra Hard remaining a persistent bottleneck. *(Due to arXiv constraints, this abstract is shortened. See PDF for the full version.)*