mlsanityData to 14 Oct 2026

TinySQL: A Progressive Text-to-SQL Dataset for Mechanistic Interpretability Research

arXiv 2503.12730 ↗First posted 17 Mar 2025cs.LG cs.AI cs.DBcited by 2

On the map: Mechanistic interpretability → Knowledge and representations inside LLMs

Abstract (arXiv)

Mechanistic interpretability research faces a gap between analyzing simple circuits in toy tasks and discovering features in large models. To bridge this gap, we propose text-to-SQL generation as an ideal task to study, as it combines the formal structure of toy tasks with real-world complexity. We introduce TinySQL, a synthetic dataset, progressing from basic to advanced SQL operations, and train models ranging from 33M to 1B parameters to establish a comprehensive testbed for interpretability. We apply multiple complementary interpretability techniques, including Edge Attribution Patching and Sparse Autoencoders, to identify minimal circuits and components supporting SQL generation. We compare circuits for different SQL subskills, evaluating their minimality, reliability, and identifiability. Finally, we conduct a layerwise logit lens analysis to reveal how models compose SQL queries across layers: from intent recognition to schema resolution to structured generation. Our work provides a robust framework for probing and comparing interpretability methods in a structured, progressively complex setting.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Knowledge and representations inside LLMs, one of 13 directions in Mechanistic interpretability. Builds on 20 of its 42 references that are in the map. First posted on arXiv 17 Mar 2025.

Builds on

20 of 42 references.

References · 20

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  3. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  4. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  5. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  6. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  7. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  8. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 688
  9. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 449
  10. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  11. 2024How to use and interpret activation patchingcited by 214
  12. 2023Attribution Patching Outperforms Automated Circuit Discoverycited by 209
8 more
  1. 2019CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databasescited by 185
  2. 2019SParC: Cross-Domain Semantic Parsing in Contextcited by 144
  3. 2020Structure-Grounded Pretraining for Text-to-SQLcited by 126
  4. 2021Towards Robustness of Text-to-SQL Models against Synonym Substitutioncited by 110
  5. 2023Physics of Language Models: Part 1, Learning Hierarchical Language Structurescited by 83
  6. 2019Text-to-SQL Generation for Question Answering on Electronic Medical Recordscited by 77
  7. 2022Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbationcited by 28
  8. 2021On the Transferability of Pre-trained Language Models: A Study from Artificial Datasetscited by 25

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Distributional Semantics Tracing: A Framework for Explaining Hallucinations in Large Language Modelscited by 3
  2. 2026How Do Language Models Understand Tables? A Mechanistic Analysis of Cell Locationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Accuracy: Introducing a Symbolic-Mechanistic Approach to Interpretable Evaluation
  2. 2025Text2SQL-Flow: A Robust SQL-Aware Data Augmentation Framework for Text-to-SQL
  3. 2026LIMIT: Less Is More for Instruction Tuning in Text-to-SQL
  4. 2025Rationalization Models for Text-to-SQL
  5. 2025DB-Explore: Automated Database Exploration and Instruction Synthesis for Text-to-SQL
  6. 2026TraceSQL: Traceable Answerability Estimation for Reference-Free Text-to-SQL Verification
  7. 2026TAHOE: Text-to-SQL with Automated Hint Optimization from Experience
  8. 2026DecoSearch: Complexity-Aware Routing and Plan-Level Repair for Text-to-SQL