# A computational model of early language acquisition from audiovisual experiences of young infants

arXiv: https://arxiv.org/abs/1906.09832 · HTML: https://mlsanity.com/paper/1906.09832/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jun 2019.

## Built on by (most cited first, 10 of 10)

- [Can phones, syllables, and words emerge as side-products of cross-situational audiovisual learning? -- A computational investigation](https://mlsanity.com/paper/2109.14200/) (2021)
- [Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Model](https://mlsanity.com/paper/2305.11435/) (2023)
- [Evaluation of Audio-Visual Alignments in Visually Grounded Speech Models](https://mlsanity.com/paper/2108.02562/) (2021)
- [Keyword localisation in untranscribed speech using visually grounded speech models](https://mlsanity.com/paper/2202.01107/) (2022)
- [Towards visually prompted keyword localisation for zero-resource spoken languages](https://mlsanity.com/paper/2210.06229/) (2022)
- [A model of early word acquisition based on realistic-scale audiovisual naming events](https://mlsanity.com/paper/2406.05259/) (2024)
- [Does Vision Accelerate Hierarchical Generalization in Neural Language Learners?](https://mlsanity.com/paper/2302.00667/) (2023)
- [A Language-agnostic Model of Child Language Acquisition](https://mlsanity.com/paper/2408.12254/) (2024)
- [Computational modeling of early language learning from acoustic speech and audiovisual input without linguistic priors](https://mlsanity.com/paper/2603.08359/) (2026)
- [Modelling word learning and recognition using visually grounded speech](https://mlsanity.com/paper/2203.06937/) (2022)

## Abstract (arXiv)

Earlier research has suggested that human infants might use statistical dependencies between speech and non-linguistic multimodal input to bootstrap their language learning before they know how to segment words from running speech. However, feasibility of this hypothesis in terms of real-world infant experiences has remained unclear. This paper presents a step towards a more realistic test of the multimodal bootstrapping hypothesis by describing a neural network model that can learn word segments and their meanings from referentially ambiguous acoustic input. The model is tested on recordings of real infant-caregiver interactions using utterance-level labels for concrete visual objects that were attended by the infant when caregiver spoke an utterance containing the name of the object, and using random visual labels for utterances during absence of attention. The results show that beginnings of lexical knowledge may indeed emerge from individually ambiguous learning scenarios. In addition, the hidden layers of the network show gradually increasing selectivity to phonetic categories as a function of layer depth, resembling models trained for phone recognition in a supervised manner.