mlsanityData to 14 Oct 2026

What Drives Recovery in Agentic Text-to-Cypher? LAST-CQ: An LLM Agent Self-Refinement Framework

arXiv 2609.12746 ↗First posted 11 Sep 2026cs.AI cs.CL cs.LGcited by 0

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

Agentic pipelines for structured-query generation are rapidly expanding, but it is unclear which part of the loop produces the gain. We use LAST-CQ -- a five-agent, training-free, execution-grounded Text-to-Cypher framework -- as an instrumented testbed, running three counterfactuals over 2,471 live-database queries and six backbones spanning three vendor scale tiers. Removing correction is worth between 3.1% aggregate execution-BLEU against the single-pass system and 12.3% against a no-refinement counterfactual (up to 80.7% for the weakest backbone). Replacing schema-grounded, LLM-synthesised feedback with raw database error strings costs almost nothing (20.9% vs. 19.9% naive exact match; <0.2% end-to-end; equivalent within ± 0.075 set-F1 by two one-sided tests). Spending the same call budget on parallel sampling degrades quality by 10-11%. What works is detecting failure and routing it to a retry, not the feedback sophistication or number of samples. LAST-CQ itself recovers 91.7% of queries that fail under single-pass generation, while a query that succeeds first time still costs exactly one LLM call. We also show that n-gram overlap on serialised results is not a bound in either direction: it over-scores against set equivalence on 65.9% of results while under-scoring against judged semantics. Finally, we calibrate our LLM judge against blind human labels and find it optimistic by 9 points.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 34 of its 56 references that are in the map. First posted on arXiv 11 Sep 2026.

Builds on

34 of 56 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Multi-Agent GraphRAG: A Text-to-Cypher Framework for Labeled Property Graphs11 Nov
  2. 2025ReFoRCE: A Text-to-SQL Agent with Self-Refinement, Consensus Enforcement, and Column Exploration2 Feb

Other references · 32

  1. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  2. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  3. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,475
  4. 2024From Local to Global: A Graph RAG Approach to Query-Focused Summarizationcited by 1,453
  5. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  6. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 688
  7. 2020A Survey on Knowledge Graphs: Representation, Acquisition and Applicationscited by 643
  8. 2020Knowledge Graphscited by 621
  9. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 477
  10. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 424
  11. 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 385
  12. 2023LEVER: Learning to Verify Language-to-Code Generation with Executioncited by 239
20 more
  1. 2023Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graphcited by 238
  2. 2023GPT4Graph: Can Large Language Models Understand Graph Structured Data ? An Empirical Evaluation and Benchmarkingcited by 210
  3. 2016Language to Logical Form with Neural Attentioncited by 158
  4. 2024MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notescited by 80
  5. 2018A Survey on Semantic Parsingcited by 62
  6. 2018Robust Text-to-SQL Generation with Execution-Guided Decodingcited by 52
  7. 2022Modern Baselines for SPARQL Semantic Parsingcited by 34
  8. 2025Large Language Models Meet Knowledge Graphs for Question Answering: Synthesis and Opportunitiescited by 31
  9. 2024Text2Cypher: Bridging Natural Language and Graph Databasescited by 30
  10. 2025ExCoT: Optimizing Reasoning for Text-to-SQL with Execution Feedbackcited by 24
  11. 2023A Universal Question-Answering Platform for Knowledge Graphscited by 17
  12. 2025ExeSQL: Self-Taught Text-to-SQL Models with Execution-Driven Bootstrapping for SQL Dialectscited by 12
  13. 2024Auto-Cypher: Improving LLMs on Cypher generation via LLM-supervised generation-verification frameworkcited by 10
  14. 2026DyTopo: Dynamic Topology Routing for Multi-Agent Reasoning via Semantic Matchingcited by 8
  15. 2025ARCS: Agentic Retrieval-Augmented Code Synthesis with Iterative Refinementcited by 7
  16. 2023S2CTrans: Building a bridge from SPARQL to Cyphercited by 6
  17. 2025Enhancing Text2Cypher with Schema Filteringcited by 6
  18. 2026PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agentscited by 4
  19. 2026CYGNET: Cypher Gate for Neural Execution Triage and Cost Containmentcited by 1
  20. 2026Achieving Precise Text-To-Cypher Via Grounded Knowledge Graph Data Generationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Agentic-SQL Revisited: Autonomy-Based Taxonomy and Empirical Benchmark Analysis for LLM Text-to-SQL
  2. 2026ACTS-SQL: Agentic and Critic-Oriented Tree-Structured SQL Correctness with Large Language Models
  3. 2026Structured Feedback Improves Repair in an LLM Agent Loop
  4. 2025BAPPA: Benchmarking Agents, Plans, and Pipelines for Automated Text-to-SQL Generation
  5. 2026AV-SQL: Decomposing Complex Text-to-SQL Queries with Agentic Views
  6. 2025A Multi-agent Text2SQL Framework using Small Language Models and Execution Feedback
  7. 2026AgentSM: Semantic Memory for Agentic Text-to-SQL
  8. 2025MARS-SQL: A multi-agent reinforcement learning framework for Text-to-SQL