# Unpacking Let Alone: Human-Scale Models Generalize to a Rare Construction in Form but not Meaning

arXiv: https://arxiv.org/abs/2506.04408 · HTML: https://mlsanity.com/paper/2506.04408/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 48 references that are in the map.
- First posted on arXiv 4 Jun 2025.

## Builds on (most cited first, 22 of 22)

- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://mlsanity.com/paper/1912.00582/) (2019)
- [What Artificial Neural Networks Can Tell Us About Human Language Acquisition](https://mlsanity.com/paper/2208.07998/) (2022)
- [Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora](https://mlsanity.com/paper/2504.08165/) (2025)
- [Cognitive Science in the era of Artificial Intelligence: A roadmap for reverse-engineering the infant language-learner](https://mlsanity.com/paper/1607.08723/) (2016)
- [Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNs](https://mlsanity.com/paper/2403.19827/) (2024)
- [minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language Models](https://mlsanity.com/paper/2203.13112/) (2022)
- [Elements of World Knowledge (EWoK): A Cognition-Inspired Framework for Evaluating Basic World Knowledge in Language Models](https://mlsanity.com/paper/2405.09605/) (2024)
- [Findings of the Second BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora](https://mlsanity.com/paper/2412.05149/) (2024)
- [Trained on 100 million words and still in shape: BERT meets British National Corpus](https://mlsanity.com/paper/2303.09859/) (2023)
- [Filtered Corpus Training (FiCT) Shows that Language Models can Generalize from Indirect Evidence](https://mlsanity.com/paper/2405.15750/) (2024)
- [CxGBERT: BERT meets Construction Grammar](https://mlsanity.com/paper/2011.04134/) (2020)
- [Neural reality of argument structure constructions](https://mlsanity.com/paper/2202.12246/) (2022)
- [A Discerning Several Thousand Judgments: GPT-3 Rates the Article + Adjective + Numeral + Noun Construction](https://mlsanity.com/paper/2301.12564/) (2023)
- [Not all layers are equally as important: Every Layer Counts BERT](https://mlsanity.com/paper/2311.02265/) (2023)
- [Both Direct and Indirect Evidence Contribute to Dative Alternation Preferences in Language Models](https://mlsanity.com/paper/2503.20850/) (2025)
- [Constructions Are So Difficult That Even Large Language Models Get Them Right for the Wrong Reasons](https://mlsanity.com/paper/2403.17760/) (2024)
- [Do Construction Distributions Shape Formal Language Learning In German BabyLMs?](https://mlsanity.com/paper/2503.11593/) (2025)
- [What Goes Into a LM Acceptability Judgment? Rethinking the Impact of Frequency and Length](https://mlsanity.com/paper/2411.02528/) (2024)
- [Constructions are Revealed in Word Distributions](https://mlsanity.com/paper/2503.06048/) (2025)
- [Beyond Memorization: Assessing Semantic Generalization in Large Language Models Using Phrasal Constructions](https://mlsanity.com/paper/2501.04661/) (2025)
- [Construction Identification and Disambiguation Using BERT: A Case Study of NPN](https://mlsanity.com/paper/2503.18751/) (2025)

## Built on by (most cited first, 4 of 4)

- [SiniticMTError: A Machine Translation Dataset with Error Annotations for Sinitic Languages](https://mlsanity.com/paper/2509.20557/) (2025)
- [Function Words as Statistical Cues for Language Learning](https://mlsanity.com/paper/2601.21191/) (2026)
- [CxMP: A Linguistic Minimal-Pair Benchmark for Evaluating Constructional Understanding in Language Models](https://mlsanity.com/paper/2602.21978/) (2026)
- [Language Models Learn Constructional Semantics, Not To Mention Syntax: Investigating LM Understanding of Paired-Focus Constructions](https://mlsanity.com/paper/2605.31586/) (2026)

## Abstract (arXiv)

Humans have a remarkable ability to acquire and understand grammatical phenomena that are seen rarely, if ever, during childhood. Recent evidence suggests that language models with human-scale pretraining data may possess a similar ability by generalizing from frequent to rare constructions. However, it remains an open question how widespread this generalization ability is, and to what extent this knowledge extends to meanings of rare constructions, as opposed to just their forms. We fill this gap by testing human-scale transformer language models on their knowledge of both the form and meaning of the (rare and quirky) English LET-ALONE construction. To evaluate our LMs we construct a bespoke synthetic benchmark that targets syntactic and semantic properties of the construction. We find that human-scale LMs are sensitive to form, even when related constructions are filtered from the dataset. However, human-scale LMs do not make correct generalizations about LET-ALONE's meaning. These results point to an asymmetry in the current architectures' sample efficiency between language form and meaning, something which is not present in human language learners.