Which Forms of Caregiver Feedback Support Grammar Learning? A Reinforcement-Learning Study of Child-Like Language Models
On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF
Abstract (arXiv)
Social interaction is central to children's language learning, but the effects of different forms of caregiver feedback are difficult to isolate in naturalistic data. We use child-like language models as controlled learners to test which forms of feedback support grammatical development. Small GPT-2-style models are pretrained on child-directed language from CHILDES, then fine-tuned with reinforcement learning using reward models trained to capture four feedback types: communicative feedback, structural alignment, semantic contingency, and affective feedback. Reward fine-tuning yields limited gains on minimal-pair evaluations, but clearer effects in free generation. Structural alignment produces the strongest improvements in grammaticality, providing a novel, plausible mechanistic account of how this feedback can support grammar learning. Communicative feedback yields more moderate gains. In contrast, semantic contingency and affective feedback do not improve grammaticality, although further analyses suggest that they may support other aspects of language learning beyond grammar. These results suggest that different forms of caregiver feedback make complementary contributions to language learning.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 14 of its 56 references that are in the map. First posted on arXiv 8 Sep 2026.
Builds on
14 of 56 references.
Closest ancestors · 1
Same direction, newest first
- 2023BabyStories: Can Reinforcement Learning Teach Baby Language Models to Write Better Stories?25 Oct
Other references · 13
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
- 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 564
- 2021A Simple Recipe for Multilingual Grammatical Error Correctioncited by 108
- 2024Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNscited by 65
- 2023Evaluating Neural Language Models as Cognitive Models of Language Acquisitioncited by 15
- 2024Babysit A Language Model From Scratch: Interactive Language Learning by Trials and Demonstrationscited by 12
- 2025Once Upon a Time: Interactive Learning for Storytelling with Small Language Modelscited by 11
- 2025Dialogue Is Not Enough to Make a Communicative BabyLM (But Neither Is Developmentally Inspired Reinforcement Learning)cited by 11
- 2025Linguistic Generalizations are not Rules: Impacts on Evaluation of LMscited by 10
- 2024Automatic Annotation of Grammaticality in Child-Caregiver Conversationscited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2023Let Me Teach You: Pedagogical Foundations of Feedback for Language Models
- 2023Can training neural language models on a curriculum with developmentally plausible data improve alignment with human reading behavior?
- 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Training
- 2020Learning Rewards from Linguistic Feedback
- 2023Interpreting Learned Feedback Patterns in Large Language Models
- 2024Reframing linguistic bootstrapping as joint inference using visually-grounded grammar induction models
- 2025Teacher Demonstrations in a BabyLM's Zone of Proximal Development for Contingent Multi-Turn Interaction
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback