# Babysit A Language Model From Scratch: Interactive Language Learning by Trials and Demonstrations

arXiv: https://arxiv.org/abs/2405.13828 · HTML: https://mlsanity.com/paper/2405.13828/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 81 references that are in the map.
- First posted on arXiv 22 May 2024.

## Builds on (most cited first, 30 of 37)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books](https://mlsanity.com/paper/1506.06724/) (2015)
- [Sequence Level Training with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06732/) (2015)
- [A Survey of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2312.14925/) (2023)
- [Experience Grounds Language](https://mlsanity.com/paper/2004.10151/) (2020)
- [Secrets of RLHF in Large Language Models Part I: PPO](https://mlsanity.com/paper/2307.04964/) (2023)
- [What Artificial Neural Networks Can Tell Us About Human Language Acquisition](https://mlsanity.com/paper/2208.07998/) (2022)
- [The MultiBERTs: BERT Reproductions for Robustness Analysis](https://mlsanity.com/paper/2106.16163/) (2021)
- [Deep Reinforcement Learning with a Natural Language Action Space](https://mlsanity.com/paper/1511.04636/) (2015)
- [Training Trajectories of Language Models Across Scales](https://mlsanity.com/paper/2212.09803/) (2022)
- [Language Understanding for Text-based Games Using Deep Reinforcement Learning](https://mlsanity.com/paper/1506.08941/) (2015)
- [Visually Grounded Neural Syntax Acquisition](https://mlsanity.com/paper/1906.02890/) (2019)
- [On the proper role of linguistically-oriented deep net analysis in linguistic theorizing](https://mlsanity.com/paper/2106.08694/) (2021)
- [Word Acquisition in Neural Language Models](https://mlsanity.com/paper/2110.02406/) (2021)
- [Language acquisition: do children and language models follow similar learning stages?](https://mlsanity.com/paper/2306.03586/) (2023)
- [Characterizing Learning Curves During Language Model Pre-Training: Learning, Forgetting, and Stability](https://mlsanity.com/paper/2308.15419/) (2023)
- [Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Models](https://mlsanity.com/paper/2205.11758/) (2022)
- [Dialog-based Language Learning](https://mlsanity.com/paper/1604.06045/) (2016)
- [Interactive Language Learning by Question Answering](https://mlsanity.com/paper/1908.10909/) (2019)
- [Exploration Based Language Learning for Text-Based Games](https://mlsanity.com/paper/2001.08868/) (2020)
- [Computational Language Acquisition with Theory of Mind](https://mlsanity.com/paper/2303.01502/) (2023)

## Built on by (most cited first, 12 of 12)

- [BabyLM Turns 3: Call for papers for the 2025 BabyLM workshop](https://mlsanity.com/paper/2502.10645/) (2025)
- [Dialogue Is Not Enough to Make a Communicative BabyLM (But Neither Is Developmentally Inspired Reinforcement Learning)](https://mlsanity.com/paper/2510.20358/) (2025)
- [Teacher Demonstrations in a BabyLM's Zone of Proximal Development for Contingent Multi-Turn Interaction](https://mlsanity.com/paper/2510.20411/) (2025)
- [The Philosophical Foundations of Growing AI Like A Child](https://mlsanity.com/paper/2502.10742/) (2025)
- [A Review of Incorporating Psychological Theories in LLMs](https://mlsanity.com/paper/2505.00003/) (2025)
- [The Mechanistic Emergence of Symbol Grounding in Language Models](https://mlsanity.com/paper/2510.13796/) (2025)
- [A Distributional Perspective on Word Learning in Neural Language Models](https://mlsanity.com/paper/2502.05892/) (2025)
- [Discourse Heuristics For Paradoxically Moral Self-Correction](https://mlsanity.com/paper/2507.00985/) (2025)
- [Playpen: An Environment for Exploring Learning Through Conversational Interaction](https://mlsanity.com/paper/2504.08590/) (2025)
- [Computer Vision Models Show Human-Like Sensitivity to Geometric and Topological Concepts](https://mlsanity.com/paper/2505.13281/) (2025)
- [Grounded Concreteness: Human-Like Concreteness Sensitivity in Vision-Language Models](https://mlsanity.com/paper/2601.18065/) (2026)
- [Which Forms of Caregiver Feedback Support Grammar Learning? A Reinforcement-Learning Study of Child-Like Language Models](https://mlsanity.com/paper/2609.08576/) (2026)

## Abstract (arXiv)

Humans are efficient language learners and inherently social creatures. Our language development is largely shaped by our social interactions, for example, the demonstration and feedback from caregivers. Contrary to human language learning, recent advancements in large language models have primarily adopted a non-interactive training paradigm, and refined pre-trained models through feedback afterward. In this work, we explore how corrective feedback from interactions influences neural language acquisition from scratch through systematically controlled experiments, assessing whether it contributes to word learning efficiency in language models. We introduce a trial-and-demonstration (TnD) learning framework that incorporates three distinct components: student trials, teacher demonstrations, and a reward conditioned on language competence at various developmental stages. Our experiments reveal that the TnD approach accelerates word acquisition for student models of equal and smaller numbers of parameters, and we highlight the significance of both trials and demonstrations. We further show that the teacher's choices of words influence students' word-specific learning efficiency, and a practice-makes-perfect effect is evident by a strong correlation between the frequency of words in trials and their respective learning curves. Our findings suggest that interactive language learning, with teacher demonstrations and active trials, can facilitate efficient word learning in language models.