mlsanityData to 7 Oct 2026

Constrained Language Models Yield Few-Shot Semantic Parsers

arXiv 2104.08768 ↗First posted 18 Apr 2021cs.CLcited by 162

Not on the map. Why?

Abstract (arXiv)

We explore the use of large pretrained language models as few-shot semantic parsers. The goal in semantic parsing is to generate a structured meaning representation given a natural language input. However, language models are trained to generate natural language. To bridge the gap, we use language models to paraphrase inputs into a controlled sublanguage resembling English that can be automatically mapped to a target meaning representation. Our results demonstrate that with only a small amount of data and very little code to convert into English-like representations, our blueprint for rapidly bootstrapping semantic parsers leads to surprisingly effective performance on multiple community tasks, greatly exceeding baseline methods also trained on the same limited data.

Key facts. Cited by 162 papers in the map (data through 7 Oct 2026). Builds on 40 of its 64 references that are in the map. First posted on arXiv 18 Apr 2021.

Builds on

40 of 64 references.

References · 40

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  5. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  6. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  7. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  8. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  9. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  10. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  11. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  12. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
28 more
  1. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 934
  2. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  3. 2020It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learnerscited by 668
  4. 2018A Corpus for Reasoning About Natural Language Grounded in Photographscited by 457
  5. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  6. 2018The Web as a Knowledge-base for Answering Complex Questionscited by 409
  7. 2017Lexically Constrained Decoding for Sequence Generation Using Grid Beam Searchcited by 234
  8. 2018Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translationcited by 181
  9. 2020GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsingcited by 171
  10. 2016Language to Logical Form with Neural Attentioncited by 158
  11. 2020Break It Down: A Question Understanding Benchmarkcited by 131
  12. 2020Task-Oriented Dialogue as Dataflow Synthesiscited by 122
  13. 2018Correcting Length Bias in Neural Machine Translationcited by 119
  14. 2016Data Recombination for Neural Semantic Parsingcited by 111
  15. 2016Guided Open Vocabulary Image Captioning with Constrained Beam Searchcited by 105
  16. 2018A Retrieve-and-Edit Framework for Predicting Structured Outputscited by 73
  17. 2020Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsingcited by 72
  18. 2020Zero-Shot Transfer Learning with Synthesized Data for Multi-Domain Dialogue State Trackingcited by 65
  19. 2021Unlocking Compositional Generalization in Pre-trained Models Using Intermediate Representationscited by 55
  20. 2019Model-based Interactive Semantic Parsing: A Unified Framework and A Text-to-SQL Case Studycited by 40
  21. 2021NL-EDIT: Correcting semantic parse errors through natural language interactioncited by 33
  22. 2018ComQA: A Community-sourced Dataset for Complex Factoid Question Answering with Paraphrase Clusterscited by 29
  23. 2020AutoQA: From Databases To QA Semantic Parsers With Only Synthetic Training Datacited by 29
  24. 2019Semantic Parsing with Dual Learningcited by 28
  25. 2019Genie: A Generator of Natural Language Semantic Parsers for Virtual Assistant Commandscited by 19
  26. 2020Unnatural Language Processing: Bridging the Gap Between Synthetic and Natural Language Datacited by 16
  27. 2016Semantic Parsing with Semi-Supervised Sequential Autoencoderscited by 11
  28. 2021Low-Resource Task-Oriented Semantic Parsing via Intrinsic Modelingcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 47; 2023: 48; 2024: 25; 2025: 17; 2026: 15.

162 papers.

Sort:

By area:

  1. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  2. 2021Learning To Retrieve Prompts for In-Context Learningcited by 641
  3. 2022PAL: Program-aided Language Modelscited by 635
  4. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  5. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 389
  6. 2022RARR: Researching and Revising What Language Models Say, Using Language Modelscited by 286
  7. 2022UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Modelscited by 231
  8. 2022Optimizing Prompts for Text-to-Image Generationcited by 204
  9. 2023Grammar-Constrained Decoding for Structured NLP Tasks without Finetuningcited by 186
  10. 2022The Unreliability of Explanations in Few-shot Prompting for Textual Reasoningcited by 169
  11. 2023Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Promptscited by 161
  12. 2022Synchromesh: Reliable code generation from pre-trained language modelscited by 156
38 more
  1. 2022Prompting Is Programming: A Query Language for Large Language Modelscited by 110
  2. 2024A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methodscited by 101
  3. 2022Compositional Semantic Parsing with Large Language Modelscited by 95
  4. 2021Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoningcited by 85
  5. 2022Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Againcited by 83
  6. 2022Complementary Explanations for Effective In-Context Learningcited by 71
  7. 2025JSONSchemaBench: A Rigorous Benchmark of Structured Outputs for Language Modelscited by 66
  8. 2022TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversationscited by 65
  9. 2023GPT-RE: In-context Learning for Relation Extraction using Large Language Modelscited by 62
  10. 2022TIARA: Multi-grained Retrieval for Robust Question Answering over Large Knowledge Basescited by 59
  11. 2023Coupling Large Language Models with Logic Programming for Robust and General Reasoning from Textcited by 59
  12. 2023Grammar Prompting for Domain-Specific Language Generation with Large Language Modelscited by 58
  13. 2024Understanding LLMs: A Comprehensive Overview from Training to Inferencecited by 58
  14. 2022Don't Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environmentscited by 57
  15. 2024Grammar-Aligned Decodingcited by 54
  16. 2023How Do In-Context Examples Affect Compositional Generalization?cited by 51
  17. 2021Few-Shot Semantic Parsing with Language Models Trained On Codecited by 50
  18. 2022Evaluating the Impact of Model Scale for Compositional Generalization in Semantic Parsingcited by 50
  19. 2022In-Context Learning for Few-Shot Dialogue State Trackingcited by 47
  20. 2023Skill-Based Few-Shot Selection for In-Context Learningcited by 46
  21. 2023Faithful Persona-based Conversational Dataset Generation with Large Language Modelscited by 45
  22. 2025Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlocited by 45
  23. 2022Retrieval-based Controllable Molecule Generationcited by 44
  24. 2023On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codexcited by 41
  25. 2022SeqZero: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Modelscited by 38
  26. 2023Data-Efficient Learning of Natural Language to Linear Temporal Logic Translators for Robot Task Specificationcited by 38
  27. 2023You Only Prompt Once: On the Capabilities of Prompt Learning on Large Language Models to Tackle Toxic Contentcited by 37
  28. 2021Communicating Natural Programs to Humans and Machinescited by 35
  29. 2022Calibrated Interpretation: Confidence Estimation in Semantic Parsingcited by 33
  30. 2024Pragmatic Instruction Following and Goal Assistance via Cooperative Language-Guided Inverse Planningcited by 28
  31. 2022On the Compositional Generalization Gap of In-Context Learningcited by 24
  32. 2023Interleaving Pre-Trained Language Models and Large Language Models for Zero-Shot NL2SQL Generationcited by 24
  33. 2022Multi-Stage Prompting for Knowledgeable Dialogue Generationcited by 22
  34. 2022Few-shot Subgoal Planning with Language Modelscited by 22
  35. 2022MultiSpider: Towards Benchmarking Multilingual Text-to-SQL Semantic Parsingcited by 22
  36. 2021The Power of Prompt Tuning for Low-Resource Semantic Parsingcited by 21
  37. 2022Dialogue Summaries as Dialogue States (DS2), Template-Guided Summarization for Few-shot Dialogue State Trackingcited by 21
  38. 2022CLASP: Few-Shot Cross-Lingual Data Augmentation for Semantic Parsingcited by 21

50 most cited of 162 All 162 (JSON)