# Exemplar Retrieval Without Overhypothesis Induction: Limits of Distributional Sequence Learning in Early Word Learning

arXiv: https://arxiv.org/abs/2604.05243 · HTML: https://mlsanity.com/paper/2604.05243/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 3 of its 7 references that are in the map.
- First posted on arXiv 6 Apr 2026.

## Builds on (most cited first, 3 of 3)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora](https://mlsanity.com/paper/2504.08165/) (2025)
- [From Shortcut to Induction Head: How Data Diversity Shapes Algorithm Selection in Transformers](https://mlsanity.com/paper/2512.18634/) (2025)

## Abstract (arXiv)

Background: Children do not simply learn that balls are round and blocks are square. They learn that shape is the kind of feature that tends to define object categories -- a second-order generalisation known as an overhypothesis [1, 2]. What kind of learning mechanism is sufficient for this inductive leap? Methods: We trained autoregressive transformer language models (3.4M-25.6M parameters) on synthetic corpora in which shape is the stable feature dimension across categories, with eight conditions controlling for alternative explanations. Results: Across 120 pre-registered runs evaluated on a 1,040-item wug test battery, every model achieved perfect first-order exemplar retrieval (100%) while second-order generalisation to novel nouns remained at chance (50-52%), a result confirmed by equivalence testing. A feature-swap diagnostic revealed that models rely on frame-to-feature template matching rather than structured noun-to-domain-to-feature abstraction. Conclusions: These results reveal a clear limitation of autoregressive distributional sequence learning under developmental-scale training conditions.