# Computational modeling of early language learning from acoustic speech and audiovisual input without linguistic priors

arXiv: https://arxiv.org/abs/2603.08359 · HTML: https://mlsanity.com/paper/2603.08359/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 119 references that are in the map.
- First posted on arXiv 9 Mar 2026.

## Builds on (most cited first, 30 of 32)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [SUPERB: Speech processing Universal PERformance Benchmark](https://mlsanity.com/paper/2105.01051/) (2021)
- [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://mlsanity.com/paper/1912.00582/) (2019)
- [An Unsupervised Autoregressive Model for Speech Representation Learning](https://mlsanity.com/paper/1904.03240/) (2019)
- [Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora](https://mlsanity.com/paper/2504.08165/) (2025)
- [Cognitive Science in the era of Artificial Intelligence: A roadmap for reverse-engineering the infant language-learner](https://mlsanity.com/paper/1607.08723/) (2016)
- [Representations of language in a model of visually grounded speech signal](https://mlsanity.com/paper/1702.01991/) (2017)
- [Word Discovery in Visually Grounded, Self-Supervised Speech Models](https://mlsanity.com/paper/2203.15081/) (2022)
- [Self-supervised language learning from raw audio: Lessons from the Zero Resource Speech Challenge](https://mlsanity.com/paper/2210.15759/) (2022)
- [Visually grounded models of spoken language: A survey of datasets, architectures and evaluation techniques](https://mlsanity.com/paper/2104.13225/) (2021)
- [BabySLM: language-acquisition-friendly benchmark of self-supervised spoken language models](https://mlsanity.com/paper/2306.01506/) (2023)
- [Encoding of phonology in a recurrent neural model of grounded speech](https://mlsanity.com/paper/1706.03815/) (2017)
- [Self-Supervised Representation Learning for Speech Using Visual Grounding and Masked Language Modeling](https://mlsanity.com/paper/2202.03543/) (2022)
- [Language learning using Speech to Image retrieval](https://mlsanity.com/paper/1909.03795/) (2019)
- [Semantic speech retrieval with a visually grounded model of untranscribed speech](https://mlsanity.com/paper/1710.01949/) (2017)
- [Self-supervised Predictive Coding Models Encode Speaker and Phonetic Information in Orthogonal Subspaces](https://mlsanity.com/paper/2305.12464/) (2023)
- [Models of Visually Grounded Speech Signal Pay Attention To Nouns: a Bilingual Experiment on English and Japanese](https://mlsanity.com/paper/1902.03052/) (2019)
- [Word Recognition, Competition, and Activation in a Model of Visually Grounded Speech](https://mlsanity.com/paper/1909.08491/) (2019)
- [DevBench: A multimodal developmental benchmark for language learning](https://mlsanity.com/paper/2406.10215/) (2024)
- [Can phones, syllables, and words emerge as side-products of cross-situational audiovisual learning? -- A computational investigation](https://mlsanity.com/paper/2109.14200/) (2021)
- [Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Model](https://mlsanity.com/paper/2305.11435/) (2023)
- [A computational model of early language acquisition from audiovisual experiences of young infants](https://mlsanity.com/paper/1906.09832/) (2019)
- [Blind phoneme segmentation with temporal prediction errors](https://mlsanity.com/paper/1608.00508/) (2016)
- [Unsupervised word segmentation and lexicon discovery using acoustic word embeddings](https://mlsanity.com/paper/1603.02845/) (2016)
- [Learning English with Peppa Pig](https://mlsanity.com/paper/2202.12917/) (2022)
- [Evaluating computational models of infant phonetic learning across languages](https://mlsanity.com/paper/2008.02888/) (2020)
- [Computational and Robotic Models of Early Language Development: A Review](https://mlsanity.com/paper/1903.10246/) (2019)
- [Analysis of Predictive Coding Models for Phonemic Representation Learning in Small Datasets](https://mlsanity.com/paper/2007.04205/) (2020)

## Built on by (most cited first, 1 of 1)

- [The Learning Objective Governs Perceptual Narrowing: A Cross-Lingual, Layer-Wise, Ten-Seed Study of Self-Supervised Speech Encoders](https://mlsanity.com/paper/2608.00507/) (2026)

## Abstract (arXiv)

Learning to understand speech appears almost effortless for typically developing infants, yet from an information-processing perspective, acquiring a language from acoustic speech is an enormous challenge. This chapter reviews recent developments in using computational models to understand early language acquisition from speech and audiovisual input. The focus is on self-supervised and visually grounded models of perceptual learning. We show how these models are becoming increasingly powerful in learning various aspects of speech without strong linguistic priors, and how many features of early language development can be explained through a shared set of learning principles-principles broadly compatible with multiple theories of language acquisition and human cognition. We also discuss how modern learning simulations are gradually becoming more realistic, both in terms of input data and in linking model behavior to empirical findings on infant language development.