# Uncontrolled Lexical Exposure Leads to Overestimation of Compositional Generalization in Pretrained Models

arXiv: https://arxiv.org/abs/2212.10769 · HTML: https://mlsanity.com/paper/2212.10769/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 54 references that are in the map.
- First posted on arXiv 21 Dec 2022.

## Builds on (most cited first, 30 of 31)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation](https://mlsanity.com/paper/2109.00859/) (2021)
- [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://mlsanity.com/paper/2105.13626/) (2021)
- [Measuring Compositional Generalization: A Comprehensive Method on Realistic Data](https://mlsanity.com/paper/1912.09713/) (2019)
- [COGS: A Compositional Generalization Challenge Based on Semantic Interpretation](https://mlsanity.com/paper/2010.05465/) (2020)
- [Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?](https://mlsanity.com/paper/2010.12725/) (2020)
- [Improving Text-to-SQL Evaluation Methodology](https://mlsanity.com/paper/1806.09029/) (2018)
- [Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP](https://mlsanity.com/paper/2112.10508/) (2021)
- [The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers](https://mlsanity.com/paper/2108.12284/) (2021)
- [Compositional Generalization in Semantic Parsing: Pre-training vs. Specialized Architectures](https://mlsanity.com/paper/2007.08970/) (2020)
- [Making Transformers Solve Compositional Tasks](https://mlsanity.com/paper/2108.04378/) (2021)
- [Meta-Learning to Compositionally Generalize](https://mlsanity.com/paper/2106.04252/) (2021)
- [Unlocking Compositional Generalization in Pre-trained Models Using Intermediate Representations](https://mlsanity.com/paper/2104.07478/) (2021)
- [Evaluating the Impact of Model Scale for Compositional Generalization in Semantic Parsing](https://mlsanity.com/paper/2205.12253/) (2022)
- [Improving Compositional Generalization with Latent Structure and Data Augmentation](https://mlsanity.com/paper/2112.07610/) (2021)
- [Systematic Generalization with Edge Transformers](https://mlsanity.com/paper/2112.00578/) (2021)
- [Learning Algebraic Recombination for Compositional Generalization](https://mlsanity.com/paper/2107.06516/) (2021)
- [Challenges in Generalization in Open Domain Question Answering](https://mlsanity.com/paper/2109.01156/) (2021)
- [Disentangled Sequence to Sequence Learning for Compositional Generalization](https://mlsanity.com/paper/2110.04655/) (2021)
- [Unobserved Local Structures Make Compositional Generalization Hard](https://mlsanity.com/paper/2201.05899/) (2022)
- [Lexicon Learning for Few-Shot Neural Sequence Modeling](https://mlsanity.com/paper/2106.03993/) (2021)
- [Are Pre-trained Convolutions Better than Pre-trained Transformers?](https://mlsanity.com/paper/2105.03322/) (2021)
- [How Much Does Tokenization Affect Neural Machine Translation?](https://mlsanity.com/paper/1812.08621/) (2018)
- [Inducing Transformer's Compositional Generalization Ability via Auxiliary Sequence Prediction Tasks](https://mlsanity.com/paper/2109.15256/) (2021)
- [SyGNS: A Systematic Generalization Testbed Based on Natural Language Semantics](https://mlsanity.com/paper/2106.01077/) (2021)
- [LAGr: Labeling Aligned Graphs for Improving Systematic Generalization in Semantic Parsing](https://mlsanity.com/paper/2110.07572/) (2021)
- [Compositional generalization in semantic parsing with pretrained transformers](https://mlsanity.com/paper/2109.15101/) (2021)
- [Do Language Models Learn Position-Role Mappings?](https://mlsanity.com/paper/2202.03611/) (2022)

## Built on by (most cited first, 26 of 26)

- [Inverse Scaling: When Bigger Isn't Better](https://mlsanity.com/paper/2306.09479/) (2023)
- [Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve](https://mlsanity.com/paper/2309.13638/) (2023)
- [Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement](https://mlsanity.com/paper/2310.08559/) (2023)
- [Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNs](https://mlsanity.com/paper/2403.19827/) (2024)
- [How Do In-Context Examples Affect Compositional Generalization?](https://mlsanity.com/paper/2305.04835/) (2023)
- [Break It Down: Evidence for Structural Compositionality in Neural Networks](https://mlsanity.com/paper/2301.10884/) (2023)
- [The Impact of Depth on Compositional Generalization in Transformer Language Models](https://mlsanity.com/paper/2310.19956/) (2023)
- [Dissociating language and thought in large language models](https://mlsanity.com/paper/2301.06627/) (2023)
- [ReCOGS: How Incidental Details of a Logical Form Overshadow an Evaluation of Semantic Interpretation](https://mlsanity.com/paper/2303.13716/) (2023)
- [Language Models as Models of Language](https://mlsanity.com/paper/2408.07144/) (2024)
- [SLOG: A Structural Generalization Benchmark for Semantic Parsing](https://mlsanity.com/paper/2310.15040/) (2023)
- [The Validity of Evaluation Results: Assessing Concurrence Across Compositionality Benchmarks](https://mlsanity.com/paper/2310.17514/) (2023)
- [Monotonic Location Attention for Length Generalization](https://mlsanity.com/paper/2305.20019/) (2023)
- [Analyzing the Inner Workings of Transformers in Compositional Generalization](https://mlsanity.com/paper/2502.15277/) (2025)
- [Differentiable Tree Operations Promote Compositional Generalization](https://mlsanity.com/paper/2306.00751/) (2023)
- [Quantifying artificial intelligence through algorithmic generalization](https://mlsanity.com/paper/2411.05943/) (2024)
- [Compositional Generalization Across Distributional Shifts with Sparse Tree Operations](https://mlsanity.com/paper/2412.14076/) (2024)
- [Is artificial intelligence still intelligence? LLMs generalize to novel adjective-noun pairs, but don't mimic the full human distribution](https://mlsanity.com/paper/2410.17482/) (2024)
- [Rapid Word Learning Through Meta In-Context Learning](https://mlsanity.com/paper/2502.14791/) (2025)
- [On the generalization capacity of neural networks during generic multimodal reasoning](https://mlsanity.com/paper/2401.15030/) (2024)
- [Harnessing Dataset Cartography for Improved Compositional Generalization in Transformers](https://mlsanity.com/paper/2310.12118/) (2023)
- [Evaluating Structural Generalization in Neural Machine Translation](https://mlsanity.com/paper/2406.13363/) (2024)
- [Sequential Compositional Generalization in Multimodal Models](https://mlsanity.com/paper/2404.12013/) (2024)
- [On the Design Space Between Transformers and Recursive Neural Nets](https://mlsanity.com/paper/2409.01531/) (2024)
- [Compositional Generalisation for Explainable Hate Speech Detection](https://mlsanity.com/paper/2506.03916/) (2025)
- [Does Episodic Memory Help Close the Lexical Frequency Gap in Sensitivity to Syntactic Contrasts? A Test Using Retrieval-Augmented Language Models](https://mlsanity.com/paper/2608.23851/) (2026)

## Abstract (arXiv)

Human linguistic capacity is often characterized by compositionality and the generalization it enables -- human learners can produce and comprehend novel complex expressions by composing known parts. Several benchmarks exploit distributional control across training and test to gauge compositional generalization, where certain lexical items only occur in limited contexts during training. While recent work using these benchmarks suggests that pretrained models achieve impressive generalization performance, we argue that exposure to pretraining data may break the aforementioned distributional control. Using the COGS benchmark of Kim and Linzen (2020), we test two modified evaluation setups that control for this issue: (1) substituting context-controlled lexical items with novel character sequences, and (2) substituting them with special tokens represented by novel embeddings. We find that both of these setups lead to lower generalization performance in T5 (Raffel et al., 2020), suggesting that previously reported results have been overestimated due to uncontrolled lexical exposure during pretraining. The performance degradation is more extreme with novel embeddings, and the degradation increases with the amount of pretraining data, highlighting an interesting case of inverse scaling.