mlsanityData to 7 Oct 2026

Archer: A Human-Labeled Text-to-SQL Dataset with Arithmetic, Commonsense and Hypothetical Reasoning

arXiv 2402.12554 ↗First posted 19 Feb 2024cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

We present Archer, a challenging bilingual text-to-SQL dataset specific to complex reasoning, including arithmetic, commonsense and hypothetical reasoning. It contains 1,042 English questions and 1,042 Chinese questions, along with 521 unique SQL queries, covering 20 English databases across 20 domains. Notably, this dataset demonstrates a significantly higher level of complexity compared to existing publicly available datasets. Our evaluation shows that Archer challenges the capabilities of current state-of-the-art models, with a high-ranked model on the Spider leaderboard achieving only 6.73% execution accuracy on Archer test set. Thus, Archer presents a significant challenge for future research in this field.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 17 of its 41 references that are in the map. First posted on arXiv 19 Feb 2024.

Builds on

17 of 41 references.

References · 17

  1. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  2. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  3. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  4. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 465
  5. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 402
  6. 2023RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQLcited by 191
  7. 2018Improving Text-to-SQL Evaluation Methodologycited by 133
  8. 2020Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic Parsingcited by 133
  9. 2021KaggleDBQA: Realistic Evaluation of Text-to-SQL Parserscited by 105
  10. 2017Learning a Neural Semantic Parser from User Feedbackcited by 98
  11. 2023Graphix-T5: Mixing Pre-Trained Transformers with Graph-Aware Layers for Text-to-SQL Parsingcited by 93
  12. 2021Knowledge-aware Zero-Shot Learning: Survey and Perspectivecited by 41
5 more
  1. 2021Zero-shot Visual Question Answering using Knowledge Graphcited by 37
  2. 2019Commonsense Properties from Query Logs and Question Answering Forumscited by 26
  3. 2023Towards Knowledge-Intensive Text-to-SQL Semantic Parsing with Formulaic Knowledgecited by 18
  4. 2018Knowledge-based Transfer Learning Explanationcited by 12
  5. 2023BUCA: A Binary Classification Approach to Unsupervised Commonsense Question Answeringcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 5; 2026: 6.

13 papers.

Sort:

By area:

  1. 2024A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going?cited by 54
  2. 2025Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?cited by 36
  3. 2024Prompting Large Language Models with Knowledge Graphs for Question Answering Involving Long-tail Factscited by 14
  4. 2025MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmarkcited by 13
  5. 2025Pi-SQL: Enhancing Text-to-SQL with Fine-Grained Guidance from Pivot Programming Languagescited by 3
  6. 2025Transduction is All You Need for Structured Data Workflowscited by 3
  7. 2025OraPlan-SQL: A Planning-Centric Framework for Complex Bilingual NL2SQL Reasoningcited by 3
  8. 2026PolySQL: Scaling Text-to-SQL Evaluation Across SQL Dialects via Automated Backend Isomorphismcited by 1
  9. 2026Dial: A Knowledge-Grounded Dialect-Specific NL2SQL Systemcited by 0
  10. 2026Process Supervision for Chain-of-Thought Reasoning via Monte Carlo Net Information Gaincited by 0
  11. 2026SOMA-SQL: Resolving Multi-Source Ambiguity in NL-to-SQL via Synthetic Log and Execution Probingcited by 0
  12. 2026SDAM: Structure-Difference-Aware Memory Evolution for Complex Text-to-SQLcited by 0
1 more
  1. 2026When Tokenization is Secretly Output Supervisioncited by 0