# Awakening Latent Grounding from Pretrained Language Models for Semantic Parsing

arXiv: https://arxiv.org/abs/2109.10540 · HTML: https://mlsanity.com/paper/2109.10540/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 56 references that are in the map.
- First posted on arXiv 22 Sep 2021.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [A Primer in BERTology: What we know about how BERT works](https://mlsanity.com/paper/2002.12327/) (2020)
- [Designing and Interpreting Probes with Control Tasks](https://mlsanity.com/paper/1909.03368/) (2019)
- [Compositional Semantic Parsing on Semi-Structured Tables](https://mlsanity.com/paper/1508.00305/) (2015)
- [What do you learn from context? Probing for sentence structure in contextualized word representations](https://mlsanity.com/paper/1905.06316/) (2019)
- [RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers](https://mlsanity.com/paper/1911.04942/) (2019)
- [Evaluating the visualization of what a Deep Neural Network has learned](https://mlsanity.com/paper/1509.06321/) (2015)
- [Linguistic Knowledge and Transferability of Contextual Representations](https://mlsanity.com/paper/1903.08855/) (2019)
- [What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models](https://mlsanity.com/paper/1907.13528/) (2019)
- [Visual7W: Grounded Question Answering in Images](https://mlsanity.com/paper/1511.03416/) (2015)
- [Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation](https://mlsanity.com/paper/1905.08205/) (2019)
- [Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic Parsing](https://mlsanity.com/paper/2012.12627/) (2020)
- [Open Sesame: Getting Inside BERT's Linguistic Knowledge](https://mlsanity.com/paper/1906.01698/) (2019)
- [Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT](https://mlsanity.com/paper/2004.14786/) (2020)
- [Relational Graph Attention Network for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2004.12362/) (2020)
- [A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualization](https://mlsanity.com/paper/1902.01069/) (2019)
- [You Impress Me: Dialogue Generation via Mutual Persona Perception](https://mlsanity.com/paper/2004.05388/) (2020)
- [TypeSQL: Knowledge-based Type-Aware Neural Text-to-SQL Generation](https://mlsanity.com/paper/1804.09769/) (2018)
- [Editing-Based SQL Query Generation for Cross-Domain Context-Dependent Questions](https://mlsanity.com/paper/1909.00786/) (2019)
- [Grounded Video Description](https://mlsanity.com/paper/1812.06587/) (2018)
- [Efficient One-Pass End-to-End Entity Linking for Questions](https://mlsanity.com/paper/2010.02413/) (2020)
- [On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL Queries](https://mlsanity.com/paper/2010.11246/) (2020)
- ["What is Relevant in a Text Document?": An Interpretable Machine Learning Approach](https://mlsanity.com/paper/1612.07843/) (2016)
- [How Far are We from Effective Context Modeling? An Exploratory Study on Semantic Parsing in Context](https://mlsanity.com/paper/2002.00652/) (2020)
- [Global Reasoning over Database Structures for Text-to-SQL Parsing](https://mlsanity.com/paper/1908.11214/) (2019)

## Built on by (most cited first, 28 of 28)

- [Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL](https://mlsanity.com/paper/2406.08426/) (2024)
- [A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going?](https://mlsanity.com/paper/2408.05109/) (2024)
- [Synthesizing Text-to-SQL Data from Weak and Strong LLMs](https://mlsanity.com/paper/2408.03256/) (2024)
- [A Survey on Text-to-SQL Parsing: Concepts, Methods, and Future Directions](https://mlsanity.com/paper/2208.13629/) (2022)
- [Before Generation, Align it! A Novel and Effective Strategy for Mitigating Hallucinations in Text-to-SQL Generation](https://mlsanity.com/paper/2405.15307/) (2024)
- [Recent Advances in Text-to-SQL: A Survey of What We Have and What We Expect](https://mlsanity.com/paper/2208.10099/) (2022)
- [Proton: Probing Schema Linking Information from Pre-trained Language Models for Text-to-SQL Parsing](https://mlsanity.com/paper/2206.14017/) (2022)
- [A Survey of Table Reasoning with Large Language Models](https://mlsanity.com/paper/2402.08259/) (2024)
- [Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbation](https://mlsanity.com/paper/2212.09994/) (2022)
- [Semantic Enhanced Text-to-SQL Parsing via Iteratively Learning Schema Linking Graph](https://mlsanity.com/paper/2208.03903/) (2022)
- [UniSAr: A Unified Structure-Aware Autoregressive Language Model for Text-to-SQL](https://mlsanity.com/paper/2203.07781/) (2022)
- [STAR: SQL Guided Pre-Training for Context-dependent Text-to-SQL Parsing](https://mlsanity.com/paper/2210.11888/) (2022)
- [Towards Knowledge-Intensive Text-to-SQL Semantic Parsing with Formulaic Knowledge](https://mlsanity.com/paper/2301.01067/) (2023)
- [Deep Learning Driven Natural Languages Text to SQL Query Conversion: A Survey](https://mlsanity.com/paper/2208.04415/) (2022)
- [SUN: Exploring Intrinsic Uncertainties in Text-to-SQL Parsers](https://mlsanity.com/paper/2209.06442/) (2022)
- [Reflection of Thought: Inversely Eliciting Numerical Reasoning in Language Models via Solving Linear Systems](https://mlsanity.com/paper/2210.05075/) (2022)
- [MURRE: Multi-Hop Table Retrieval with Removal for Open-Domain Text-to-SQL](https://mlsanity.com/paper/2402.10666/) (2024)
- [Know What I don't Know: Handling Ambiguous and Unanswerable Questions for Text-to-SQL](https://mlsanity.com/paper/2212.08902/) (2022)
- [A Comparative Analysis of Conversational Large Language Models in Knowledge-Based Text Generation](https://mlsanity.com/paper/2402.01495/) (2024)
- [DAC: Decomposed Automation Correction for Text-to-SQL](https://mlsanity.com/paper/2408.08779/) (2024)
- [Unveiling the Black Box of PLMs with Semantic Anchors: Towards Interpretable Neural Semantic Parsing](https://mlsanity.com/paper/2210.01425/) (2022)
- [Uncovering and Categorizing Social Biases in Text-to-SQL](https://mlsanity.com/paper/2305.16253/) (2023)
- [Benchmarking Diverse-Modal Entity Linking with Generative Models](https://mlsanity.com/paper/2305.17337/) (2023)
- [Translate First Reorder Later: Leveraging Monotonicity in Semantic Parsing](https://mlsanity.com/paper/2210.04878/) (2022)
- [Language Model Pre-Training with Sparse Latent Typing](https://mlsanity.com/paper/2210.12582/) (2022)
- [Linking-Enhanced Pre-Training for Table Semantic Parsing](https://mlsanity.com/paper/2111.09486/) (2021)
- [Controllable Data Augmentation for Context-Dependent Text-to-SQL](https://mlsanity.com/paper/2304.13902/) (2023)
- [RepCali: High Efficient Fine-tuning Via Representation Calibration in Latent Space for Pre-trained Language Models](https://mlsanity.com/paper/2505.08463/) (2025)

## Abstract (arXiv)

Recent years pretrained language models (PLMs) hit a success on several downstream tasks, showing their power on modeling language. To better understand and leverage what PLMs have learned, several techniques have emerged to explore syntactic structures entailed by PLMs. However, few efforts have been made to explore grounding capabilities of PLMs, which are also essential. In this paper, we highlight the ability of PLMs to discover which token should be grounded to which concept, if combined with our proposed erasing-then-awakening approach. Empirical studies on four datasets demonstrate that our approach can awaken latent grounding which is understandable to human experts, even if it is not exposed to such labels during training. More importantly, our approach shows great potential to benefit downstream semantic parsing models. Taking text-to-SQL as a case study, we successfully couple our approach with two off-the-shelf parsers, obtaining an absolute improvement of up to 9.8%.