# Learning to Recombine and Resample Data for Compositional Generalization

arXiv: https://arxiv.org/abs/2010.03706 · HTML: https://mlsanity.com/paper/2010.03706/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Oct 2020.

## Built on by (most cited first, 30 of 65)

- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [Compositional Semantic Parsing with Large Language Models](https://mlsanity.com/paper/2209.15003/) (2022)
- [Tailor: Generating and Perturbing Text with Semantic Controls](https://mlsanity.com/paper/2107.07150/) (2021)
- [Compositional Generalization from First Principles](https://mlsanity.com/paper/2307.05596/) (2023)
- [Neural Data Augmentation via Example Extrapolation](https://mlsanity.com/paper/2102.01335/) (2021)
- [How Do In-Context Examples Affect Compositional Generalization?](https://mlsanity.com/paper/2305.04835/) (2023)
- [Evaluating the Impact of Model Scale for Compositional Generalization in Semantic Parsing](https://mlsanity.com/paper/2205.12253/) (2022)
- [Improving Compositional Generalization with Latent Structure and Data Augmentation](https://mlsanity.com/paper/2112.07610/) (2021)
- [Systematic Generalization with Edge Transformers](https://mlsanity.com/paper/2112.00578/) (2021)
- [SeqZero: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Models](https://mlsanity.com/paper/2205.07381/) (2022)
- [Learning Algebraic Recombination for Compositional Generalization](https://mlsanity.com/paper/2107.06516/) (2021)
- [Sequence-to-Sequence Learning with Latent Neural Grammars](https://mlsanity.com/paper/2109.01135/) (2021)
- [Disentangled Sequence to Sequence Learning for Compositional Generalization](https://mlsanity.com/paper/2110.04655/) (2021)
- [Unobserved Local Structures Make Compositional Generalization Hard](https://mlsanity.com/paper/2201.05899/) (2022)
- [Lexicon Learning for Few-Shot Neural Sequence Modeling](https://mlsanity.com/paper/2106.03993/) (2021)
- [Finding needles in a haystack: Sampling Structurally-diverse Training Sets from Synthetic Data for Compositional Generalization](https://mlsanity.com/paper/2109.02575/) (2021)
- [How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Composition](https://mlsanity.com/paper/2207.07730/) (2022)
- [Neural-Logic Human-Object Interaction Detection](https://mlsanity.com/paper/2311.09817/) (2023)
- [Improving Generalization in Language Model-Based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-Based Techniques](https://mlsanity.com/paper/2305.17378/) (2023)
- [A Philosophical Introduction to Language Models -- Part I: Continuity With Classic Debates](https://mlsanity.com/paper/2401.03910/) (2024)
- [Revisiting the Compositional Generalization Abilities of Neural Sequence Models](https://mlsanity.com/paper/2203.07402/) (2022)
- [Inducing Transformer's Compositional Generalization Ability via Auxiliary Sequence Prediction Tasks](https://mlsanity.com/paper/2109.15256/) (2021)
- [Improving Compositional Generalization with Self-Training for Data-to-Text Generation](https://mlsanity.com/paper/2110.08467/) (2021)
- [ExeDec: Execution Decomposition for Compositional Generalization in Neural Program Synthesis](https://mlsanity.com/paper/2307.13883/) (2023)
- [SUBS: Subtree Substitution for Compositional Semantic Parsing](https://mlsanity.com/paper/2205.01538/) (2022)
- [Temporal Representation Alignment: Successor Features Enable Emergent Compositionality in Robot Instruction Following](https://mlsanity.com/paper/2502.05454/) (2025)
- [On The Ingredients of an Effective Zero-shot Semantic Parser](https://mlsanity.com/paper/2110.08381/) (2021)
- [Neural-Symbolic Recursive Machine for Systematic Generalization](https://mlsanity.com/paper/2210.01603/) (2022)
- [Generating Data for Symbolic Language with Large Language Models](https://mlsanity.com/paper/2305.13917/) (2023)
- [Grammar-Based Grounded Lexicon Learning](https://mlsanity.com/paper/2202.08806/) (2022)

## Abstract (arXiv)

Flexible neural sequence models outperform grammar- and automaton-based counterparts on a variety of tasks. However, neural models perform poorly in settings requiring compositional generalization beyond the training data -- particularly to rare or unseen subsequences. Past work has found symbolic scaffolding (e.g. grammars or automata) essential in these settings. We describe R&R, a learned data augmentation scheme that enables a large category of compositional generalizations without appeal to latent symbolic structure. R&R has two components: recombination of original training examples via a prototype-based generative model and resampling of generated examples to encourage extrapolation. Training an ordinary neural sequence model on a dataset augmented with recombined and resampled examples significantly improves generalization in two language processing problems -- instruction following (SCAN) and morphological analysis (SIGMORPHON 2018) -- where R&R enables learning of new constructions and tenses from as few as eight initial examples.