mlsanityData to 14 Oct 2026

Diverse Demonstrations Improve In-context Compositional Generalization

arXiv 2212.06800 ↗First posted 13 Dec 2022cs.CLcited by 139

Not on the map. Why?

Abstract (arXiv)

In-context learning has shown great success in i.i.d semantic parsing splits, where the training and test sets are drawn from the same distribution. In this setup, models are typically prompted with demonstrations that are similar to the input utterance. However, in the setup of compositional generalization, where models are tested on outputs with structures that are absent from the training set, selecting similar demonstrations is insufficient, as often no example will be similar enough to the input. In this work, we propose a method to select diverse demonstrations that aims to collectively cover all of the structures required in the output program, in order to encourage the model to generalize to new structures from these demonstrations. We empirically show that combining diverse demonstrations with in-context learning substantially improves performance across three compositional generalization semantic parsing datasets in the pure in-context learning setup and when combined with finetuning.

Key facts. Cited by 139 papers in the map (data through 14 Oct 2026). Builds on 40 of its 62 references that are in the map. First posted on arXiv 13 Dec 2022.

Builds on

40 of 62 references.

References · 40

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  7. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  8. 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
  9. 2021MetaICL: Learning to Learn In Contextcited by 454
  10. 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 388
  11. 2018AllenNLP: A Deep Semantic Natural Language Processing Platformcited by 347
  12. 2019Measuring Compositional Generalization: A Comprehensive Method on Realistic Datacited by 284
28 more
  1. 2022Selective Annotation Makes Language Models Better Few-Shot Learnerscited by 243
  2. 2021Meta-learning via Language Model In-context Tuningcited by 164
  3. 2020Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?cited by 147
  4. 2018Improving Text-to-SQL Evaluation Methodologycited by 135
  5. 2020Task-Oriented Dialogue as Dataflow Synthesiscited by 122
  6. 2019Compositional generalization through meta sequence-to-sequence learningcited by 99
  7. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 98
  8. 2020Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architecturescited by 95
  9. 2022Compositional Semantic Parsing with Large Language Modelscited by 95
  10. 2022Complementary Explanations for Effective In-Context Learningcited by 71
  11. 2021Meta-Learning to Compositionally Generalizecited by 62
  12. 2022Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Datacited by 61
  13. 2022Evaluating the Impact of Model Scale for Compositional Generalization in Semantic Parsingcited by 50
  14. 2021Improving Compositional Generalization with Latent Structure and Data Augmentationcited by 46
  15. 2021Joint Passage Ranking for Diverse Multi-Answer Retrievalcited by 43
  16. 2021Controllable Semantic Parsing via Retrieval Augmentationcited by 35
  17. 2022Unobserved Local Structures Make Compositional Generalization Hardcited by 29
  18. 2022A Well-Composed Text is Half Done! Composition Sampling for Diverse Conditional Generationcited by 28
  19. 2021Finding needles in a haystack: Sampling Structurally-diverse Training Sets from Synthetic Data for Compositional Generalizationcited by 27
  20. 2019Don't paraphrase, detect! Rapid and Effective Data Collection for Semantic Parsingcited by 24
  21. 2022On the Compositional Generalization Gap of In-Context Learningcited by 24
  22. 2021Determinantal Beam Searchcited by 18
  23. 2021RETRONLU: Retrieval Augmented Task-Oriented Semantic Parsingcited by 15
  24. 2022Generate-and-Retrieve: use your predictions to improve retrieval for semantic parsingcited by 15
  25. 2022Submodlib: A Submodular Optimization Librarycited by 8
  26. 2022Structurally Diverse Sampling for Sample-Efficient Training and Comprehensive Evaluationcited by 8
  27. 2022Simplifying Semantic Annotations of SMCalFlowcited by 7
  28. 2022Diverse Multi-Answer Retrieval with Determinantal Point Processescited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 38; 2024: 44; 2025: 36; 2026: 20.

139 papers.

Sort:

By area:

  1. 2023A Survey of Large Language Modelscited by 2,576
  2. 2022A Survey on In-context Learningcited by 721
  3. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 503
  4. 2023What can Large Language Models do in chemistry? A comprehensive benchmark on eight taskscited by 185
  5. 2023Compositional Exemplars for In-context Learningcited by 144
  6. 2024In-Context Learning with Long-Context Models: An In-Depth Explorationcited by 128
  7. 2023Finding Support Examples for In-Context Learningcited by 104
  8. 2023When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunitiescited by 90
  9. 2023Testing the General Deductive Reasoning Capacity of Large Language Models Using OOD Examplescited by 84
  10. 2023In-Context Learning with Iterative Demonstration Selectioncited by 74
  11. 2024In-context Learning with Retrieved Demonstrations for Language Models: A Surveycited by 70
  12. 2023Coverage-based Example Selection for In-Context Learningcited by 59
38 more
  1. 2023How to Prompt LLMs for Text-to-SQL: A Study in Zero-shot, Single-domain, and Cross-domain Settingscited by 58
  2. 2023Grammar Prompting for Domain-Specific Language Generation with Large Language Modelscited by 58
  3. 2023OpenICL: An Open-Source Framework for In-context Learningcited by 51
  4. 2023How Do In-Context Examples Affect Compositional Generalization?cited by 51
  5. 2023Skill-Based Few-Shot Selection for In-Context Learningcited by 46
  6. 2023Large Language Model-Aware In-Context Learning for Code Generationcited by 43
  7. 2023Small Models are Valuable Plug-ins for Large Language Modelscited by 42
  8. 2023Self-ICL: Zero-Shot In-Context Learning with Self-Generated Demonstrationscited by 42
  9. 2024Lifelong Knowledge Editing for LLMs with Retrieval-Augmented Continuous Prompt Learningcited by 40
  10. 2024Do Large Language Models Have Compositional Ability? An Investigation into Limitations and Scalabilitycited by 40
  11. 2023MoT: Memory-of-Thought Enables ChatGPT to Self-Improvecited by 38
  12. 2024Efficient Prompt Optimization Through the Lens of Best Arm Identificationcited by 37
  13. 2024Prompt Optimization with EASE? Efficient Ordering-aware Automated Selection of Exemplarscited by 37
  14. 2023How to Configure Good In-Context Sequence for Visual Question Answeringcited by 33
  15. 2024LLMs Are In-Context Bandit Reinforcement Learnerscited by 29
  16. 2024In-Context Learning Enables Robot Action Prediction in LLMscited by 27
  17. 2024Retrieval Augmented Fact Verification by Synthesizing Contrastive Argumentscited by 25
  18. 2023RetICL: Sequential Retrieval of In-Context Examples with Reinforcement Learningcited by 24
  19. 2023Improving Generalization in Language Model-Based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-Based Techniquescited by 24
  20. 2023Selective Demonstrations for Cross-domain Text-to-SQLcited by 21
  21. 2025From Few to Many: Self-Improving Many-Shot Reasoners Through Iterative Optimization and Generationcited by 21
  22. 2023LMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Promptingcited by 17
  23. 2024DemoRank: Selecting Effective Demonstrations for Large Language Models in Ranking Taskcited by 17
  24. 2024Enhancing In-context Learning via Linear Probe Calibrationcited by 16
  25. 2024On the Noise Robustness of In-Context Learning for Text Generationcited by 16
  26. 2023Cost-Effective In-Context Learning for Entity Resolution: A Design Space Explorationcited by 15
  27. 2023Overview of the PromptCBLUE Shared Task in CHIP2023cited by 14
  28. 2024Learning to Retrieve Iteratively for In-Context Learningcited by 13
  29. 2023Bring Your Own KG: Self-Supervised Program Synthesis for Zero-Shot KGQAcited by 11
  30. 2024Demonstration Augmentation for Zero-shot In-context Learningcited by 11
  31. 2025Is Relevance Propagated from Retriever to Generator in RAG?cited by 11
  32. 2023Diverse Retrieval-Augmented In-Context Learning for Dialogue State Trackingcited by 10
  33. 2023Leveraging Code to Improve In-context Learning for Semantic Parsingcited by 10
  34. 2024Improving Demonstration Diversity by Human-Free Fusing for Text-to-SQLcited by 10
  35. 2024Unleashing the Power of Large Language Models in Zero-shot Relation Extraction via Self-Promptingcited by 10
  36. 2025Optimizing Decomposition for Optimal Claim Verificationcited by 10
  37. 2023Distilling Rule-based Knowledge into Large Language Modelscited by 9
  38. 2024NICE: To Optimize In-Context Examples or Not?cited by 9

50 most cited of 139 All 139 (JSON)