mlsanityData to 14 Oct 2026

Diverse In-Context Example Selection After Decomposing Programs and Aligned Utterances Improves Semantic Parsing

arXiv 2504.03541 ↗First posted 4 Apr 2025cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

LLMs are increasingly used as seq2seq translators from natural language utterances to structured programs, a process called semantic interpretation. Unlike atomic labels or token sequences, programs are naturally represented as abstract syntax trees (ASTs). Such structured representation raises novel issues related to the design and selection of in-context examples (ICEs) presented to the LLM. We focus on decomposing the pool of available ICE trees into fragments, some of which may be better suited to solving the test instance. Next, we propose how to use (additional invocations of) an LLM with prompted syntax constraints to automatically map the fragments to corresponding utterances. Finally, we adapt and extend a recent method for diverse ICE selection to work with whole and fragmented ICE instances. We evaluate our system, SCUD4ICL, on popular diverse semantic parsing benchmarks, showing visible accuracy gains from our proposed decomposed diverse demonstration method. Benefits are particularly notable for smaller LLMs, ICE pools having larger labeled trees, and programs in lower resource languages.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 29 of its 36 references that are in the map. First posted on arXiv 4 Apr 2025.

Builds on

29 of 36 references.

References · 29

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  4. 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 896
  5. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  6. 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
  7. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 477
  8. 2023Larger language models do in-context learning differentlycited by 394
  9. 2019Measuring Compositional Generalization: A Comprehensive Method on Realistic Datacited by 284
  10. 2022Selective Annotation Makes Language Models Better Few-Shot Learnerscited by 243
  11. 2020MTOP: A Comprehensive Multilingual Task-Oriented Semantic Parsing Benchmarkcited by 158
  12. 2020Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?cited by 147
17 more
  1. 2023Compositional Exemplars for In-context Learningcited by 144
  2. 2022Diverse Demonstrations Improve In-context Compositional Generalizationcited by 139
  3. 2018Improving Text-to-SQL Evaluation Methodologycited by 135
  4. 2023Are Emergent Abilities in Large Language Models just In-Context Learning?cited by 109
  5. 2023Coverage-based Example Selection for In-Context Learningcited by 59
  6. 2023How Many Demonstrations Do You Need for In-context Learning?cited by 52
  7. 2023How Do In-Context Examples Affect Compositional Generalization?cited by 51
  8. 2022Evaluating the Impact of Model Scale for Compositional Generalization in Semantic Parsingcited by 50
  9. 2024Why Larger Language Models Do In-context Learning Differently?cited by 45
  10. 2023CodeScope: An Execution-based Multilingual Multitask Multidimensional Benchmark for Evaluating LLMs on Code Understanding and Generationcited by 39
  11. 2023Divide and Prompt: Chain of Thought Prompting for Text-to-SQLcited by 20
  12. 2023ExeDec: Execution Decomposition for Compositional Generalization in Neural Program Synthesiscited by 17
  13. 2023Question Decomposition Tree for Answering Complex Questions over Knowledge Basescited by 15
  14. 2023Semantic Decomposition of Question and SQL for Text-to-SQL Parsingcited by 14
  15. 2023Leveraging Code to Improve In-context Learning for Semantic Parsingcited by 10
  16. 2024DataFrame QA: A Universal LLM Framework on DataFrame Question Answering Without Data Exposurecited by 6
  17. 2023HPE:Answering Complex Questions over Text by Hybrid Question Parsing and Executioncited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Contrast Matters: Understanding Robustness of In-Context Fine-Tuning to Target-Context Relatednesscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2019Learning Semantic Parsers from Denotations with Latent Structured Alignments and Abstract Programs
  2. 2025Step-by-step Instructions and a Simple Tabular Output Format Improve the Dependency Parsing Accuracy of LLMs
  3. 2026SemBench: A Universal Semantic Framework for LLM Evaluation
  4. 2025Evaluation of Finetuned LLMs in AMR Parsing
  5. 2025Large-Scale Constraint Generation -- Can LLMs Parse Hundreds of Constraints?
  6. 2023Evaluating statistical language models as pragmatic reasoners
  7. 2026Reasoning about In-Context Samples for Machine-Translation
  8. 2025SR-LLM: Rethinking the Structured Representation in Large Language Model