mlsanityData to 7 Oct 2026

Unmasking Database Vulnerabilities: Zero-Knowledge Schema Inference Attacks in Text-to-SQL Systems

arXiv 2406.14545 ↗First posted 20 Jun 2024cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

Text-to-SQL systems empower users to interact with databases using natural language, automatically translating queries into executable SQL code. However, their reliance on database schema information for SQL generation exposes them to significant security vulnerabilities, particularly schema inference attacks that can lead to unauthorized data access or manipulation. In this paper, we introduce a novel zero-knowledge framework for reconstructing the underlying database schema of text-to-SQL models without any prior knowledge of the database. Our approach systematically probes text-to-SQL models with specially crafted questions and leverages a surrogate GPT-4 model to interpret the outputs, effectively uncovering hidden schema elements -- including tables, columns, and data types. We demonstrate that our method achieves high accuracy in reconstructing table names, with F1 scores of up to .99 for generative models and .78 for fine-tuned models, underscoring the severity of schema leakage risks. We also show that our attack can steal prompt information in non-text-to-SQL models. Furthermore, we propose a simple protection mechanism for generative models and empirically show its limitations in mitigating these attacks.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 40 of its 78 references that are in the map. First posted on arXiv 20 Jun 2024.

Builds on

40 of 78 references.

References · 40

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2013Intriguing properties of neural networkscited by 5,114
  6. 2020Extracting Training Data from Large Language Modelscited by 2,144
  7. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  8. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  9. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  10. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
  11. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  12. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
28 more
  1. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  2. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 465
  3. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
  4. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 420
  5. 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 395
  6. 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 378
  7. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 376
  8. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 329
  9. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  10. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 233
  11. 2023RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQLcited by 191
  12. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
  13. 2020GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsingcited by 171
  14. 2023C3: Zero-shot Text-to-SQL with ChatGPTcited by 159
  15. 2020Contextualized Perturbation for Textual Adversarial Attackcited by 136
  16. 2020Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic Parsingcited by 133
  17. 2024On Protecting the Data Privacy of Large Language Models (LLMs): A Surveycited by 91
  18. 2019A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualizationcited by 88
  19. 2022RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQLcited by 85
  20. 2018TypeSQL: Knowledge-based Type-Aware Neural Text-to-SQL Generationcited by 81
  21. 2019Text-to-SQL Generation for Question Answering on Electronic Medical Recordscited by 76
  22. 2022Recent Advances in Text-to-SQL: A Survey of What We Have and What We Expectcited by 47
  23. 2024SQLBench: A Comprehensive Evaluation for Text-to-SQL Capabilities of Large Language Modelscited by 32
  24. 2024Teach LLMs to Phish: Stealing Private Information from Language Modelscited by 29
  25. 2023Improving Generalization in Language Model-Based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-Based Techniquescited by 23
  26. 2023Towards Understanding the Generalization of Medical Text-to-SQL Models and Datasetscited by 9
  27. 2023BERT Lost Patience Won't Be Robust to Adversarial Slowdowncited by 4
  28. 2022On the Security Vulnerabilities of Text-to-SQL Modelscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

  1. 2025MaskSQL: Safeguarding Privacy for LLM-Based Text-to-SQL via Abstractioncited by 3
  2. 2025A Multi-Agent Framework for Mitigating Dialect Biases in Privacy Policy Question-Answering Systemscited by 2
  3. 2026Beyond Text-to-SQL: An Agentic LLM System for Governed Enterprise Analytics APIscited by 1
  4. 2026Agents That Know Too Much: A Data-Centric Survey of Privacy in LLM Agentscited by 0
  5. 2026Policy-Conditioned Constrained Decoding for Column-Level Access Control in Text-to-SQLcited by 0
  6. 2026TrustDABench: Benchmarking Reliability and Robustness of LLMs for Structured Data Analysiscited by 0