mlsanityData to 7 Oct 2026

Awakening Latent Grounding from Pretrained Language Models for Semantic Parsing

arXiv 2109.10540 ↗First posted 22 Sep 2021cs.CL cs.AIcited by 28

Not on the map. Why?

Abstract (arXiv)

Recent years pretrained language models (PLMs) hit a success on several downstream tasks, showing their power on modeling language. To better understand and leverage what PLMs have learned, several techniques have emerged to explore syntactic structures entailed by PLMs. However, few efforts have been made to explore grounding capabilities of PLMs, which are also essential. In this paper, we highlight the ability of PLMs to discover which token should be grounded to which concept, if combined with our proposed erasing-then-awakening approach. Empirical studies on four datasets demonstrate that our approach can awaken latent grounding which is understandable to human experts, even if it is not exposed to such labels during training. More importantly, our approach shows great potential to benefit downstream semantic parsing models. Taking text-to-SQL as a case study, we successfully couple our approach with two off-the-shelf parsers, obtaining an absolute improvement of up to 9.8%.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 38 of its 56 references that are in the map. First posted on arXiv 22 Sep 2021.

Builds on

38 of 56 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2019Language Models as Knowledge Bases?cited by 1,922
  6. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  7. 2020A Primer in BERTology: What we know about how BERT workscited by 807
  8. 2019Designing and Interpreting Probes with Control Taskscited by 608
  9. 2015Compositional Semantic Parsing on Semi-Structured Tablescited by 509
  10. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 436
  11. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 420
  12. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 416
26 more
  1. 2019Linguistic Knowledge and Transferability of Contextual Representationscited by 382
  2. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 317
  3. 2015Visual7W: Grounded Question Answering in Imagescited by 315
  4. 2019Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representationcited by 210
  5. 2020Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic Parsingcited by 133
  6. 2019Open Sesame: Getting Inside BERT's Linguistic Knowledgecited by 119
  7. 2020Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERTcited by 109
  8. 2020Relational Graph Attention Network for Aspect-based Sentiment Analysiscited by 104
  9. 2019A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualizationcited by 88
  10. 2020You Impress Me: Dialogue Generation via Mutual Persona Perceptioncited by 85
  11. 2018TypeSQL: Knowledge-based Type-Aware Neural Text-to-SQL Generationcited by 81
  12. 2019Editing-Based SQL Query Generation for Cross-Domain Context-Dependent Questionscited by 80
  13. 2018Grounded Video Descriptioncited by 78
  14. 2020Efficient One-Pass End-to-End Entity Linking for Questionscited by 76
  15. 2020On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL Queriescited by 59
  16. 2016"What is Relevant in a Text Document?": An Interpretable Machine Learning Approachcited by 53
  17. 2020How Far are We from Effective Context Modeling? An Exploratory Study on Semantic Parsing in Contextcited by 53
  18. 2019Global Reasoning over Database Structures for Text-to-SQL Parsingcited by 43
  19. 2020Can neural networks acquire a structural bias from raw linguistic data?cited by 35
  20. 2020Is Supervised Syntactic Parsing Beneficial for Language Understanding? An Empirical Investigationcited by 34
  21. 2019Grammar-based Neural Text-to-SQL Generationcited by 23
  22. 2017Learning Structured Natural Language Representations for Semantic Parsingcited by 15
  23. 2020"What Do You Mean by That?" A Parser-Independent Interactive Approach for Enhancing Text-to-SQLcited by 11
  24. 2018Mixing Context Granularities for Improved Entity Linking on Question Answering Data across Entity Categoriescited by 9
  25. 2017Entity Linking for Queries by Searching Wikipedia Sentencescited by 5
  26. 2020A Tale of Two Linkings: Dynamically Gating between Schema Linking and Structural Linking for Text-to-SQL Parsingcited by 4

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 14; 2023: 4; 2024: 8; 2025: 1.

28 papers.

Sort:

By area:

  1. 2024Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQLcited by 118
  2. 2024A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going?cited by 54
  3. 2024Synthesizing Text-to-SQL Data from Weak and Strong LLMscited by 53
  4. 2022A Survey on Text-to-SQL Parsing: Concepts, Methods, and Future Directionscited by 51
  5. 2024Before Generation, Align it! A Novel and Effective Strategy for Mitigating Hallucinations in Text-to-SQL Generationcited by 49
  6. 2022Recent Advances in Text-to-SQL: A Survey of What We Have and What We Expectcited by 47
  7. 2022Proton: Probing Schema Linking Information from Pre-trained Language Models for Text-to-SQL Parsingcited by 30
  8. 2024A Survey of Table Reasoning with Large Language Modelscited by 29
  9. 2022Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbationcited by 28
  10. 2022Semantic Enhanced Text-to-SQL Parsing via Iteratively Learning Schema Linking Graphcited by 22
  11. 2022UniSAr: A Unified Structure-Aware Autoregressive Language Model for Text-to-SQLcited by 20
  12. 2022STAR: SQL Guided Pre-Training for Context-dependent Text-to-SQL Parsingcited by 19
16 more
  1. 2023Towards Knowledge-Intensive Text-to-SQL Semantic Parsing with Formulaic Knowledgecited by 18
  2. 2022Deep Learning Driven Natural Languages Text to SQL Query Conversion: A Surveycited by 11
  3. 2022SUN: Exploring Intrinsic Uncertainties in Text-to-SQL Parserscited by 7
  4. 2022Reflection of Thought: Inversely Eliciting Numerical Reasoning in Language Models via Solving Linear Systemscited by 7
  5. 2024MURRE: Multi-Hop Table Retrieval with Removal for Open-Domain Text-to-SQLcited by 7
  6. 2022Know What I don't Know: Handling Ambiguous and Unanswerable Questions for Text-to-SQLcited by 6
  7. 2024A Comparative Analysis of Conversational Large Language Models in Knowledge-Based Text Generationcited by 6
  8. 2024DAC: Decomposed Automation Correction for Text-to-SQLcited by 6
  9. 2022Unveiling the Black Box of PLMs with Semantic Anchors: Towards Interpretable Neural Semantic Parsingcited by 5
  10. 2023Uncovering and Categorizing Social Biases in Text-to-SQLcited by 5
  11. 2023Benchmarking Diverse-Modal Entity Linking with Generative Modelscited by 5
  12. 2022Translate First Reorder Later: Leveraging Monotonicity in Semantic Parsingcited by 3
  13. 2022Language Model Pre-Training with Sparse Latent Typingcited by 2
  14. 2021Linking-Enhanced Pre-Training for Table Semantic Parsingcited by 1
  15. 2023Controllable Data Augmentation for Context-Dependent Text-to-SQLcited by 1
  16. 2025RepCali: High Efficient Fine-tuning Via Representation Calibration in Latent Space for Pre-trained Language Modelscited by 1