# Towards Developmentally Plausible Rewards: Communicative Success as a Learning Signal for Interactive Language Models

arXiv: https://arxiv.org/abs/2505.05970 · HTML: https://mlsanity.com/paper/2505.05970/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 81 references that are in the map.
- First posted on arXiv 9 May 2025.

## Builds on (most cited first, 25 of 25)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling](https://mlsanity.com/paper/2304.01373/) (2023)
- [Know What You Don't Know: Unanswerable Questions for SQuAD](https://mlsanity.com/paper/1806.03822/) (2018)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts](https://mlsanity.com/paper/2102.08981/) (2021)
- [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://mlsanity.com/paper/1912.00582/) (2019)
- [UnifiedQA: Crossing Format Boundaries With a Single QA System](https://mlsanity.com/paper/2005.00700/) (2020)
- [What Artificial Neural Networks Can Tell Us About Human Language Acquisition](https://mlsanity.com/paper/2208.07998/) (2022)
- [Multi-agent Communication meets Natural Language: Synergies between Functional and Structural Language Learning](https://mlsanity.com/paper/2005.07064/) (2020)
- [Call for Papers -- The BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpus](https://mlsanity.com/paper/2301.11796/) (2023)
- [Countering Language Drift with Seeded Iterated Learning](https://mlsanity.com/paper/2003.12694/) (2020)
- [On the proper role of linguistically-oriented deep net analysis in linguistic theorizing](https://mlsanity.com/paper/2106.08694/) (2021)
- [Word Acquisition in Neural Language Models](https://mlsanity.com/paper/2110.02406/) (2021)
- [Language Models as Models of Language](https://mlsanity.com/paper/2408.07144/) (2024)
- [Not all layers are equally as important: Every Layer Counts BERT](https://mlsanity.com/paper/2311.02265/) (2023)
- [CLIMB: Curriculum Learning for Infant-inspired Model Building](https://mlsanity.com/paper/2311.08886/) (2023)
- [Computational Language Acquisition with Theory of Mind](https://mlsanity.com/paper/2303.01502/) (2023)
- [What can linguistics and deep learning contribute to each other?](https://mlsanity.com/paper/1809.04179/) (2018)
- [Dynamic population-based meta-learning for multi-agent communication with natural language](https://mlsanity.com/paper/2110.14241/) (2021)
- [Towards Pragmatic Production Strategies for Natural Language Generation Tasks](https://mlsanity.com/paper/2210.12828/) (2022)
- [Language Evolution with Deep Learning](https://mlsanity.com/paper/2403.11958/) (2024)
- [A Distributional Perspective on Word Learning in Neural Language Models](https://mlsanity.com/paper/2502.05892/) (2025)

## Built on by (most cited first, 4 of 4)

- [Once Upon a Time: Interactive Learning for Storytelling with Small Language Models](https://mlsanity.com/paper/2509.15714/) (2025)
- [Dialogue Is Not Enough to Make a Communicative BabyLM (But Neither Is Developmentally Inspired Reinforcement Learning)](https://mlsanity.com/paper/2510.20358/) (2025)
- [Child-Directed Language Does Not Consistently Boost Syntax Learning in Language Models](https://mlsanity.com/paper/2505.23689/) (2025)
- [Which Forms of Caregiver Feedback Support Grammar Learning? A Reinforcement-Learning Study of Child-Like Language Models](https://mlsanity.com/paper/2609.08576/) (2026)

## Abstract (arXiv)

We propose a method for training language models in an interactive setting inspired by child language acquisition. In our setting, a speaker attempts to communicate some information to a listener in a single-turn dialogue and receives a reward if communicative success is achieved. Unlike earlier related work using image--caption data for interactive reference games, we operationalize communicative success in a more abstract language-only question--answering setting. First, we present a feasibility study demonstrating that our reward provides an indirect signal about grammaticality. Second, we conduct experiments using reinforcement learning to fine-tune language models. We observe that cognitively plausible constraints on the communication channel lead to interpretable changes in speaker behavior. However, we do not yet see improvements on linguistic evaluations from our training regime. We outline potential modifications to the task design and training configuration that could better position future work to use our methodology to observe the benefits of interaction on language learning in computational cognitive models.