# Baby Scale: Investigating Models Trained on Individual Children's Language Input

arXiv: https://arxiv.org/abs/2603.29522 · HTML: https://mlsanity.com/paper/2603.29522/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 52 references that are in the map.
- First posted on arXiv 31 Mar 2026.

## Builds on (most cited first, 12 of 12)

- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Scaling Laws for Autoregressive Generative Modeling](https://mlsanity.com/paper/2010.14701/) (2020)
- [TinyStories: How Small Can Language Models Be and Still Speak Coherent English?](https://mlsanity.com/paper/2305.07759/) (2023)
- [Mission: Impossible Language Models](https://mlsanity.com/paper/2401.06416/) (2024)
- [GPT or BERT: why not both?](https://mlsanity.com/paper/2410.24159/) (2024)
- [Maximize Your Data's Potential: Enhancing LLM Accuracy with Two-Phase Pretraining](https://mlsanity.com/paper/2412.15285/) (2024)
- [Visual Grounding Helps Learn Word Meanings in Low-Data Regimes](https://mlsanity.com/paper/2310.13257/) (2023)
- [The BabyView dataset: High-resolution egocentric videos of infants' and young children's everyday experiences](https://mlsanity.com/paper/2406.10447/) (2024)
- [gzip Predicts Data-dependent Scaling Laws](https://mlsanity.com/paper/2405.16684/) (2024)
- [Neural Scaling Laws Rooted in the Data Distribution](https://mlsanity.com/paper/2412.07942/) (2024)
- [Challenges in Automated Processing of Speech from Child Wearables: The Case of Voice Type Classifier](https://mlsanity.com/paper/2506.11074/) (2025)

## Built on by (most cited first, 2 of 2)

- [HalluWorld: A Controlled Benchmark for Hallucination via Reference World Models](https://mlsanity.com/paper/2605.19341/) (2026)
- [To Memorize or to Retrieve: Scaling the Interaction Between Pretraining and Retrieval](https://mlsanity.com/paper/2604.00715/) (2026)

## Abstract (arXiv)

Modern language models (LMs) must be trained on many orders of magnitude more words of training data than human children receive before they begin to produce useful behavior. Assessing the nature and origins of this "data gap" requires benchmarking LMs on human-scale datasets to understand how linguistic knowledge emerges from children's natural training data. Using transcripts from the BabyView dataset (videos from children ages 6-36 months), we investigate (1) scaling performance at child-scale data regimes, (2) variability in model performance across datasets from different children's experiences and linguistic predictors of dataset quality, and (3) relationships between model and child language learning outcomes. LMs trained on child data show acceptable scaling for grammar tasks, but lower scaling on semantic and world knowledge tasks than models trained on synthetic data; we also observe substantial variability on data from different children. Beyond dataset size, performance is most associated with a combination of distributional and interactional linguistic features, broadly consistent with what makes high-quality input for child language development. Finally, model likelihoods for individual words correlate with children's learning of those words, suggesting that properties of child-directed input may influence both model learning and human language development. Overall, understanding what properties make language data efficient for learning can enable more powerful small-scale language models while also shedding light on human language acquisition.