# Content Enhanced BERT-based Text-to-SQL Generation

arXiv: https://arxiv.org/abs/1910.07179 · HTML: https://mlsanity.com/paper/1910.07179/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Oct 2019.

## Built on by (most cited first, 16 of 16)

- [Recent Advances in Text-to-SQL: A Survey of What We Have and What We Expect](https://mlsanity.com/paper/2208.10099/) (2022)
- [Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbation](https://mlsanity.com/paper/2212.09994/) (2022)
- [Dual Reader-Parser on Hybrid Textual and Tabular Evidence for Open Domain Question Answering](https://mlsanity.com/paper/2108.02866/) (2021)
- [Uni-Parser: Unified Semantic Parser for Question Answering on Knowledge Base and Database](https://mlsanity.com/paper/2211.05165/) (2022)
- [From Natural Language to SQL: Review of LLM-based Text-to-SQL Systems](https://mlsanity.com/paper/2410.01066/) (2024)
- [SUQL: Conversational Search over Structured and Unstructured Data with Large Language Models](https://mlsanity.com/paper/2311.09818/) (2023)
- [Deep Learning Driven Natural Languages Text to SQL Query Conversion: A Survey](https://mlsanity.com/paper/2208.04415/) (2022)
- [CLTR: An End-to-End, Transformer-Based System for Cell Level Table Retrieval and Table Question Answering](https://mlsanity.com/paper/2106.04441/) (2021)
- [Pay More Attention to History: A Context Modelling Strategy for Conversational Text-to-SQL](https://mlsanity.com/paper/2112.08735/) (2021)
- [KGValidator: A Framework for Automatic Validation of Knowledge Graph Construction](https://mlsanity.com/paper/2404.15923/) (2024)
- [A Copy Mechanism for Handling Knowledge Base Elements in SPARQL Neural Machine Translation](https://mlsanity.com/paper/2211.10271/) (2022)
- [MCTS-SQL: Light-Weight LLMs can Master the Text-to-SQL through Monte Carlo Tree Search](https://mlsanity.com/paper/2501.16607/) (2025)
- [EICopilot: Search and Explore Enterprise Information over Large-scale Knowledge Graphs with LLM-driven Agents](https://mlsanity.com/paper/2501.13746/) (2025)
- [Turing: an Accurate and Interpretable Multi-Hypothesis Cross-Domain Natural Language Database Interface](https://mlsanity.com/paper/2106.04559/) (2021)
- [Translating Natural Language Queries to SQL Using the T5 Model](https://mlsanity.com/paper/2312.12414/) (2023)
- [dIR -- Discrete Information Retrieval: Conversational Search over Unstructured (and Structured) Data with Large Language Models](https://mlsanity.com/paper/2312.13264/) (2023)

## Abstract (arXiv)

We present a simple methods to leverage the table content for the BERT-based model to solve the text-to-SQL problem. Based on the observation that some of the table content match some words in question string and some of the table header also match some words in question string, we encode two addition feature vector for the deep model. Our methods also benefit the model inference in testing time as the tables are almost the same in training and testing time. We test our model on the WikiSQL dataset and outperform the BERT-based baseline by 3.7% in logic form and 3.7% in execution accuracy and achieve state-of-the-art.