# Do Construction Distributions Shape Formal Language Learning In German BabyLMs?

arXiv: https://arxiv.org/abs/2503.11593 · HTML: https://mlsanity.com/paper/2503.11593/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 107 references that are in the map.
- First posted on arXiv 14 Mar 2025.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://mlsanity.com/paper/1912.00582/) (2019)
- [The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations](https://mlsanity.com/paper/1511.02301/) (2015)
- [The Shape of Learning Curves: a Review](https://mlsanity.com/paper/2103.10948/) (2021)
- [Cognitive Science in the era of Artificial Intelligence: A roadmap for reverse-engineering the infant language-learner](https://mlsanity.com/paper/1607.08723/) (2016)
- [Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNs](https://mlsanity.com/paper/2403.19827/) (2024)
- [Cross-Linguistic Syntactic Evaluation of Word Prediction Models](https://mlsanity.com/paper/2005.00187/) (2020)
- [Findings of the Second BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora](https://mlsanity.com/paper/2412.05149/) (2024)
- [Filtered Corpus Training (FiCT) Shows that Language Models can Generalize from Indirect Evidence](https://mlsanity.com/paper/2405.15750/) (2024)
- [BabySLM: language-acquisition-friendly benchmark of self-supervised spoken language models](https://mlsanity.com/paper/2306.01506/) (2023)
- [[Call for Papers] The 2nd BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpus](https://mlsanity.com/paper/2404.06214/) (2024)
- [Is Child-Directed Speech Effective Training Data for Language Models?](https://mlsanity.com/paper/2408.03617/) (2024)
- [Language Models as Models of Language](https://mlsanity.com/paper/2408.07144/) (2024)
- [BabyLM Turns 3: Call for papers for the 2025 BabyLM workshop](https://mlsanity.com/paper/2502.10645/) (2025)
- [Construction Grammar Provides Unique Insight into Neural Language Models](https://mlsanity.com/paper/2302.02178/) (2023)
- [XCOMPS: A Multilingual Benchmark of Conceptual Minimal Pairs](https://mlsanity.com/paper/2502.19737/) (2025)
- [From Babble to Words: Pre-Training Language Models on Continuous Streams of Phonemes](https://mlsanity.com/paper/2410.22906/) (2024)
- [Linguistic Generalizations are not Rules: Impacts on Evaluation of LMs](https://mlsanity.com/paper/2502.13195/) (2025)
- [Recurrent babbling: evaluating the acquisition of grammar from limited input data](https://mlsanity.com/paper/2010.04637/) (2020)
- [Emergent Abilities in Reduced-Scale Generative Language Models](https://mlsanity.com/paper/2404.02204/) (2024)
- [Subword models struggle with word learning, but surprisal hides it](https://mlsanity.com/paper/2502.12835/) (2025)
- [Constructions are Revealed in Word Distributions](https://mlsanity.com/paper/2503.06048/) (2025)
- [Small Language Models Also Work With Small Vocabularies: Probing the Linguistic Abilities of Grapheme- and Phoneme-Based Baby Llamas](https://mlsanity.com/paper/2410.01487/) (2024)
- [What Should Baby Models Read? Exploring Sample-Efficient Data Composition on Model Performance](https://mlsanity.com/paper/2411.06672/) (2024)

## Built on by (most cited first, 11 of 11)

- [Dialogue Is Not Enough to Make a Communicative BabyLM (But Neither Is Developmentally Inspired Reinforcement Learning)](https://mlsanity.com/paper/2510.20358/) (2025)
- [BabyBabelLM: A Multilingual Benchmark of Developmentally Plausible Training Data](https://mlsanity.com/paper/2510.10159/) (2025)
- [Child-Directed Language Does Not Consistently Boost Syntax Learning in Language Models](https://mlsanity.com/paper/2505.23689/) (2025)
- [Unpacking Let Alone: Human-Scale Models Generalize to a Rare Construction in Form but not Meaning](https://mlsanity.com/paper/2506.04408/) (2025)
- [Discourse Heuristics For Paradoxically Moral Self-Correction](https://mlsanity.com/paper/2507.00985/) (2025)
- [Learning from Child-Directed Speech in Two-Language Scenarios: A French-English Case Study](https://mlsanity.com/paper/2603.12906/) (2026)
- [CAIT: A Syntactic Parsing Toolkit for Child-Adult InTeractions](https://mlsanity.com/paper/2605.19718/) (2026)
- [BabyLM's First Constructions: Causal probing provides a signal of learning](https://mlsanity.com/paper/2506.02147/) (2025)
- [Understanding Syntactic Generalization in Structure-inducing Language Models](https://mlsanity.com/paper/2508.07969/) (2025)
- [Language Models Learn Constructional Semantics, Not To Mention Syntax: Investigating LM Understanding of Paired-Focus Constructions](https://mlsanity.com/paper/2605.31586/) (2026)
- [Child-directed speech facilitates production, not comprehension, in BabyLMs](https://mlsanity.com/paper/2606.01045/) (2026)

## Abstract (arXiv)

We analyze the influence of utterance-level construction distributions in German child-directed/child-available speech on the resulting word-level, syntactic and semantic competence (and their underlying learning trajectories) in small LMs, which we train on a novel collection of developmentally plausible language data for German. We find that trajectories are surprisingly robust for markedly different distributions of constructions in the training data, which have little effect on final accuracies and almost no effect on global learning trajectories. While syntax learning benefits from more complex utterances, word-level learning culminates in better scores with more fragmentary utterances. We argue that LMs trained on developmentally plausible data can contribute to debates on how conducive different kinds of linguistic stimuli are to language learning.