mlsanityData to 14 Oct 2026

Compositional Semantic Parsing with Large Language Models

arXiv 2209.15003 ↗First posted 29 Sep 2022cs.CL cs.AIcited by 95

Not on the map. Why?

Cited by 30 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Humans can reason compositionally when presented with new tasks. Previous research shows that appropriate prompting techniques enable large language models (LLMs) to solve artificial compositional generalization tasks such as SCAN. In this work, we identify additional challenges in more realistic semantic parsing tasks with larger vocabulary and refine these prompting techniques to address them. Our best method is based on least-to-most prompting: it decomposes the problem using prompting-based syntactic parsing, then uses this decomposition to select appropriate exemplars and to sequentially generate the semantic parse. This method allows us to set a new state of the art for CFQ while requiring only 1% of the training data used by traditional approaches. Due to the general nature of our approach, we expect similar efforts will lead to new results in other tasks and domains, especially for knowledge-intensive applications.

Key facts. Cited by 95 papers in the map (data through 14 Oct 2026). Builds on 45 of its 62 references that are in the map. First posted on arXiv 29 Sep 2022.

Builds on

45 of 62 references.

References · 45

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  7. 2022Emergent Abilities of Large Language Modelscited by 2,512
  8. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  9. 2020Extracting Training Data from Large Language Modelscited by 2,189
  10. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  11. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  12. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
33 more
  1. 2019Measuring Compositional Generalization: A Comprehensive Method on Realistic Datacited by 284
  2. 2020COGS: A Compositional Generalization Challenge Based on Semantic Interpretationcited by 263
  3. 2021Constrained Language Models Yield Few-Shot Semantic Parserscited by 163
  4. 2019Good-Enough Compositional Data Augmentationcited by 158
  5. 2020Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?cited by 147
  6. 2019Thieves on Sesame Street! Model Extraction of BERT-based APIscited by 139
  7. 2022Natural Language to Code Translation with Executioncited by 124
  8. 2020A Benchmark for Systematic Generalization in Grounded Language Understandingcited by 108
  9. 2017Building machines that adapt and compute like brainscited by 99
  10. 2019Compositional generalization through meta sequence-to-sequence learningcited by 99
  11. 2020Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architecturescited by 95
  12. 2020Compositional Generalization via Neural-Symbolic Stack Machinescited by 80
  13. 2020Span-based Semantic Parsing for Compositional Generalizationcited by 77
  14. 2020Learning Compositional Rules via Neural Program Synthesiscited by 74
  15. 2020Learning to Recombine and Resample Data for Compositional Generalizationcited by 65
  16. 2019CLOSURE: Assessing Systematic Generalization of CLEVR Modelscited by 63
  17. 2021Meta-Learning to Compositionally Generalizecited by 62
  18. 2019Compositional generalization in a deep seq2seq model by separating syntax and semanticscited by 56
  19. 2021Unlocking Compositional Generalization in Pre-trained Models Using Intermediate Representationscited by 55
  20. 2020Compositional Generalization by Learning Analytical Expressionscited by 53
  21. 2018Rearranging the Familiar: Testing Compositional Generalization in Recurrent Networkscited by 51
  22. 2021Few-Shot Semantic Parsing with Language Models Trained On Codecited by 50
  23. 2022Evaluating the Impact of Model Scale for Compositional Generalization in Semantic Parsingcited by 50
  24. 2019Compositional Generalization for Primitive Substitutionscited by 48
  25. 2021Improving Compositional Generalization with Latent Structure and Data Augmentationcited by 46
  26. 2022SeqZero: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Modelscited by 38
  27. 2021Learning Algebraic Recombination for Compositional Generalizationcited by 31
  28. 2021Sequence-to-Sequence Learning with Latent Neural Grammarscited by 30
  29. 2021Lexicon Learning for Few-Shot Neural Sequence Modelingcited by 28
  30. 2020Hierarchical Poset Decoding for Compositional Generalization in Languagecited by 26
  31. 2019Compositional Generalization in Image Captioningcited by 23
  32. 2022Measuring and Improving Compositional Generalization in Text-to-SQL via Component Alignmentcited by 16
  33. 2021Grounded Graph Decoding Improves Compositional Generalization in Question Answeringcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 9; 2023: 41; 2024: 27; 2025: 11; 2026: 7.

95 papers.

Sort:

By area:

  1. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  2. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  3. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  4. 2022Towards Reasoning in Large Language Models: A Surveycited by 596
  5. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 487
  6. 2023Augmented Language Models: a Surveycited by 403
  7. 2023Benchmarking Large Language Models in Retrieval-Augmented Generationcited by 312
  8. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problemscited by 260
  9. 2022Is GPT-3 a Good Data Annotator?cited by 208
  10. 2023Active Prompting with Chain-of-Thought for Large Language Modelscited by 154
  11. 2022Diverse Demonstrations Improve In-context Compositional Generalizationcited by 139
  12. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 135
38 more
  1. 2023Deductive Verification of Chain-of-Thought Reasoningcited by 124
  2. 2023Teaching Arithmetic to Small Transformerscited by 123
  3. 2022Large Language Models are few(1)-shot Table Reasonerscited by 116
  4. 2024Self-Discover: Large Language Models Self-Compose Reasoning Structurescited by 110
  5. 2023Large Language Models as Analogical Reasonerscited by 107
  6. 2024Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervisioncited by 96
  7. 2023Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Modelscited by 92
  8. 2024In-Context Language Learning: Architectures and Algorithmscited by 89
  9. 2023Crawling the Internal Knowledge-Base of Language Modelscited by 85
  10. 2024BadChain: Backdoor Chain-of-Thought Prompting for Large Language Modelscited by 80
  11. 2024In-context Learning with Retrieved Demonstrations for Language Models: A Surveycited by 70
  12. 2024Demystifying Chains, Trees, and Graphs of Thoughtscited by 63
  13. 2023Towards Better Chain-of-Thought Prompting Strategies: A Surveycited by 62
  14. 2023Synthetic Prompting: Generating Chain-of-Thought Demonstrations for Large Language Modelscited by 61
  15. 2023Coverage-based Example Selection for In-Context Learningcited by 59
  16. 2023Coupling Large Language Models with Logic Programming for Robust and General Reasoning from Textcited by 59
  17. 2022Transcending Scaling Laws with 0.1% Extra Computecited by 57
  18. 2023How Do In-Context Examples Affect Compositional Generalization?cited by 51
  19. 2023On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codexcited by 41
  20. 2024SciInstruct: a Self-Reflective Instruction Annotated Dataset for Training Scientific Language Modelscited by 39
  21. 2024Efficient Data Collection for Robotic Manipulation via Compositional Generalizationcited by 37
  22. 2023Neural Graph Reasoning: Complex Logical Query Answering Meets Graph Databasescited by 35
  23. 2023XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languagescited by 34
  24. 2023Sparse Universal Transformercited by 32
  25. 2024Divide and Translate: Compositional First-Order Logic Translation and Verification for Complex Logical Reasoningcited by 27
  26. 2023ReCOGS: How Incidental Details of a Logical Form Overshadow an Evaluation of Semantic Interpretationcited by 24
  27. 2023Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Webcited by 24
  28. 2023ExeDec: Execution Decomposition for Compositional Generalization in Neural Program Synthesiscited by 17
  29. 2024Retrieval-style In-Context Learning for Few-shot Hierarchical Text Classificationcited by 16
  30. 2023Iterated Decomposition: Improving Science Q&A by Supervising Reasoning Processescited by 15
  31. 2024Structure Guided Prompt: Instructing Large Language Model in Multi-Step Reasoning by Exploring Graph Structure of the Textcited by 14
  32. 2024Limits of Transformer Language Models on Learning to Compose Algorithmscited by 13
  33. 2022Neural-Symbolic Recursive Machine for Systematic Generalizationcited by 12
  34. 2024Progressive Compositionality in Text-to-Image Generative Modelscited by 12
  35. 2023World Models for Math Story Problemscited by 11
  36. 2023Bring Your Own KG: Self-Supervised Program Synthesis for Zero-Shot KGQAcited by 11
  37. 2023Natural Language Decomposition and Interpretation of Complex Utterancescited by 10
  38. 2024Potential and Limitations of LLMs in Capturing Structured Semantics: A Case Study on SRLcited by 10

50 most cited of 95 All 95 (JSON)