mlsanityData to 7 Oct 2026

MAGIC: Generating Self-Correction Guideline for In-Context Text-to-SQL

arXiv 2406.12692 ↗First posted 18 Jun 2024cs.CL cs.AI cs.DBcited by 34

Not on the map. Why?

Abstract (arXiv)

Self-correction in text-to-SQL is the process of prompting large language model (LLM) to revise its previously incorrectly generated SQL, and commonly relies on manually crafted self-correction guidelines by human experts that are not only labor-intensive to produce but also limited by the human ability in identifying all potential error patterns in LLM responses. We introduce MAGIC, a novel multi-agent method that automates the creation of the self-correction guideline. MAGIC uses three specialized agents: a manager, a correction, and a feedback agent. These agents collaborate on the failures of an LLM-based method on the training set to iteratively generate and refine a self-correction guideline tailored to LLM mistakes, mirroring human processes but without human involvement. Our extensive experiments show that MAGIC's guideline outperforms expert human's created ones. We empirically find out that the guideline produced by MAGIC enhances the interpretability of the corrections made, providing insights in analyzing the reason behind the failures and successes of LLMs in self-correction. All agent interactions are publicly available at https://huggingface.co/datasets/microsoft/MAGIC.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 20 of its 29 references that are in the map. First posted on arXiv 18 Jun 2024.

Builds on

20 of 29 references.

References · 20

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  2. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  3. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  4. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  5. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  6. 2023Teaching Large Language Models to Self-Debugcited by 963
  7. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
  8. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  9. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 465
  10. 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 378
  11. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 236
  12. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 230
8 more
  1. 2019Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representationcited by 210
  2. 2024CodeS: Towards Building Open-source Language Models for Text-to-SQLcited by 183
  3. 2024CHESS: Contextual Harnessing for Efficient SQL Synthesiscited by 164
  4. 2023C3: Zero-shot Text-to-SQL with ChatGPTcited by 159
  5. 2023OpenAgents: An Open Platform for Language Agents in the Wildcited by 136
  6. 2024MCS-SQL: Leveraging Multiple Prompts and Multiple-Choice Selection For Text-to-SQL Generationcited by 103
  7. 2024Before Generation, Align it! A Novel and Effective Strategy for Mitigating Hallucinations in Text-to-SQL Generationcited by 49
  8. 2024SQLBench: A Comprehensive Evaluation for Text-to-SQL Capabilities of Large Language Modelscited by 32

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 17; 2026: 11.

34 papers.

Sort:

By area:

  1. 2024The Death of Schema Linking? Text-to-SQL in the Age of Well-Reasoned Language Modelscited by 67
  2. 2025Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AIcited by 39
  3. 2025Beyond Prompt Content: Enhancing LLM Performance via Content-Format Integrated Prompt Optimizationcited by 15
  4. 2025SHARE: An SLM-based Hierarchical Action CorREction Assistant for Text-to-SQLcited by 15
  5. 2025ExCyTIn-Bench: Evaluating LLM agents on Cyber Threat Investigationcited by 15
  6. 2025Understanding, Detecting, and Repairing Real-World In-Context-Learning-Based Text-to-SQL Errorscited by 14
  7. 2025SQLens: An End-to-End Framework for Error Detection and Correction in Text-to-SQLcited by 10
  8. 2025Uncovering the Impact of Chain-of-Thought Reasoning for Direct Preference Optimization: Lessons from Text-to-SQLcited by 9
  9. 2024PTD-SQL: Partitioning and Targeted Drilling with LLMs in Text-to-SQLcited by 7
  10. 2024DAC: Decomposed Automation Correction for Text-to-SQLcited by 6
  11. 2024Leveraging Prior Experience: An Expandable Auxiliary Knowledge Base for Text-to-SQLcited by 5
  12. 2026PV-SQL: Synergizing Database Probing and Rule-based Verification for Text-to-SQL Agentscited by 5
22 more
  1. 2025SAFE-SQL: Self-Augmented In-Context Learning with Fine-grained Example Selection for Text-to-SQLcited by 4
  2. 2025Exploring the Landscape of Text-to-SQL with Large Language Models: Progresses, Challenges and Opportunitiescited by 4
  3. 2026Memo-SQL: Structured Decomposition and Experience-Driven Self-Correction for Training-Free NL2SQLcited by 3
  4. 2024Context-Aware SQL Error Correction Using Few-Shot Learning -- A Novel Approach Based on NLQ, Error, and SQL Similaritycited by 2
  5. 2025Dynamic Context-Aware Prompt Recommendation for Domain-Specific AI Applicationscited by 2
  6. 2025Toward Real-World Table Agents: Capabilities, Workflows, and Design Principles for LLM-based Table Intelligencecited by 2
  7. 2025End-to-End Text-to-SQL with Dataset Selection: Leveraging LLMs for Adaptive Query Generationcited by 2
  8. 2026ErrorLLM: Modeling SQL Errors for Text-to-SQL Refinementcited by 1
  9. 2026NL2SQLBench: A Modular Benchmarking Framework for LLM-Enabled NL2SQL Solutionscited by 1
  10. 2026Knowing When to Stop: Predicting Execution-Consistency Convergence in Text-to-SQLcited by 1
  11. 2024The Role of Accuracy and Validation Effectiveness in Conversational Business Analyticscited by 0
  12. 2025SQLCritic: Correcting Text-to-SQL Generation via Clause-wise Criticcited by 0
  13. 2025GradeSQL: Test-Time Inference with Outcome Reward Models for Text-to-SQL Generation from Large Language Modelscited by 0
  14. 2025GEMMA-SQL: A Novel Text-to-SQL Model Based on Large Language Modelscited by 0
  15. 2025SCARE: A Benchmark for SQL Correction and Question Answerability Classification for Reliable EHR Question Answeringcited by 0
  16. 2025Bridging Global Intent with Local Details: A Hierarchical Representation Approach for Semantic Validation in Text-to-SQLcited by 0
  17. 2026SQL Query Engine: A Self-Healing LLM Pipeline for Natural Language to PostgreSQL Translationcited by 0
  18. 2026ZAS-SQL: Distilling Rules from Failures for Zero-Shot Text-to-SQLcited by 0
  19. 2026DecoSearch: Complexity-Aware Routing and Plan-Level Repair for Text-to-SQLcited by 0
  20. 2026Beyond Static Rules: Automated Discovery of Latent Vulnerabilities in Text-to-SQLcited by 0
  21. 2026Causal Episodic Memory for Feedback-Driven Agent Repaircited by 0
  22. 2026TraceSQL: Traceable Answerability Estimation for Reference-Free Text-to-SQL Verificationcited by 0